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EXECUTIVE  SUMMARY 


The  Department  of  Defense  (DOD)  is  responsible  for  aequiring  capabilities  for  the 
submarine  force  to  support  a  myriad  of  missions.  Historically  systems  were  acquired  and 
deployed  to  support  a  specific  capability.  Command,  Control,  Communications, 
Computers  and  Intelligence  (C4I)  capabilities  within  the  submarine  force  have  evolved 
over  the  last  century  as  new  technologies  increase  access  to  the  RF  spectrum  and 
available  bandwidth.  As  systems  evolve  and  interoperability  becomes  more  critical,  these 
are  being  integrated  into  systems  of  systems  (SOS)  to  provide  capabilities  that  were 
previously  not  available.  This  thesis  used  a  case  study  approach  to  examine  the  Common 
Submarine  Radio  Room  (CSRR)  as  a  SOS.  The  following  recommendations,  or  learning 
principles,  where  identified  as  applicable  to  the  development  and  management  of  a 
system  of  systems.  These  are: 

•  Clearly  define  the  requirements  for  the  entire  SOS  life  cycle. 

•  Avoid  building  a  SOS  before  defining  the  architecture. 

•  The  design  of  an  acknowledged  SOS  must  be  shared  to  the  maximum 
extent  practicable. 

•  System  of  systems  must  control  interfaces. 

•  Allow  the  SOS  to  go  fast  when  possible,  otherwise  go  slow. 

•  Account  for  all  of  the  “ilities”  when  developing  the  system  of  systems 
design. 

•  Consider  how  the  SOS  will  be  tested. 

•  Acknowledge  SOSs  can  change  unexpectedly. 

•  Understand  perfect  is  the  enemy  of  good  enough. 

•  Building  a  SOS  requires  building  effective  relationships. 

•  Regardless  of  what  the  SOS  is  built  for  it  must  be  able  to  support  the 
customers.  Keep  them  in  mind. 

•  Effective  SOSs  require  effective  teams  that  are  engaged,  motivated,  and 
productive. 

The  research  determined  CSRR  exhibited  the  characteristics  of  an  acknowledged 
SOS.  Common  Submarine  Radio  Room  is  made  up  of  a  number  of  independent  systems 
capable  of  operating  independently  and  has  their  own  requirements,  funding,  and 


management.  These  systems  have  their  own  engineering  and  sustainment  approaehes. 
Eaeh  system  is  fully  operational  within  its  established  requirements  but  additional 
eapabilities  are  not  fully  realized  until  they  are  integrated  into  a  SOS  (Vaneman  2012). 
As  a  SOS,  CSRR  provides  redundancy  in  several  ways.  If  a  communications  path  is  not 
available  another  can  be  selected.  If  there  is  a  network  failure,  alternate  means  to  reroute 
or  restore  network  management  exist.  Centralized  control  and  management  provides 
more  efficient  use  of  resources  and  improved  situational  awareness. 

System  of  systems  design  and  implementation  require  a  more  holistic  view. 
Developing  and  managing  a  mission  or  platform  SOS  extends  beyond  the  activities 
involved  for  a  single  system.  Managing  a  SOS  is  a  complex  endeavor  as  competing 
demands  of  the  individual  systems  must  be  addressed  and  balanced  against  the 
requirements  and  objectives  of  whole  SOS.  Working  with  an  acknowledged  SOS,  such  as 
CSRR,  means  changes  to  the  constituent  systems  must  be  evaluated  and  integrated  to 
avoid  a  disruption  or  degradation  of  the  whole  SOS  capability.  New  capabilities  that 
result  from  integrating  several  systems  into  a  SOS  can  create  confusion  as  to  who  owns 
these  new  capabilities.  Program  managers  have  responsibility  for  their  specific  system 
whereas  most  SOS  have  no  assigned  manager.  Most  programs  are  acquired  using  clearly 
defined  capability  requirements.  Systems  of  systems  requirements  and  characteristics  can 
be  more  complex  and  amorphous.  A  SOS  program  with  an  assigned  manager  must  work 
continuously  with  all  of  the  individual  programs  to  minimize  the  impact  of  one  program 
attempting  to  optimize  at  the  expense  of  the  others.  Depending  on  the  systems  involved 
and  the  type  of  SOS,  programmatic  and  systems  engineering  decisions  may  occur  at 
lower  levels  that  are  not  in  the  best  interests  of  the  overall  SOS.  Understanding  the 
characteristics  of  a  SOS  and  the  engineering  principles  involved  are  key  factors  to 
successfully  delivering  operational  capabilities  from  a  group  of  individual  systems.  Only 
recently  has  DOD  acknowledged  acquisition  of  SOS  capability  requires  a  much  more 
holistic  approach  (Director,  Systems  and  Software  Engineering  2008). 

This  thesis  researched  the  following  questions  to  regarding  CSRR. 

1 .  What  is  CSRR  and  what  characteristics  classify  it  is  as  a  SOS? 


2.  What  are  the  benefits  and  ehallenges  of  developing,  designing,  produeing, 
deploying,  and  sustaining  CSRR  as  a  SOS? 

3.  What  best  praetiees  have  been  identified  and  implemented  in  the  CSRR 
program  and  what  benefits  have  been  realized  in  terms  of  cost, 
performance,  and  schedule? 

4.  What  lessons  learned  can  be  applied  to  future  versions  of  CSRR  and 
common  radio  room  (CRR)  for  surface  combatants? 

The  research  questions  were  bounded  to  examine  the  history  leading  up  to  CSRR 
to  understand  how  evolving  requirements  and  capabilities  led  to  its  development.  The 
organizational  structure  of  the  CSRR  program  and  stakeholder  relationships,  management 
of  an  SOS  architecture  and  the  benefits  and  drawbacks,  initiatives  to  improve  cost, 
schedule,  and  performance  were  also  examined.  Last,  the  ability  to  meet  future  mission 
requirements  was  evaluated. 

The  methodology  used  a  Friedman  and  Sage  (2003)  framework  for  providing  the 
necessary  background  and  context  to  understand  the  activities  involved  in  managing  a 
SOS  and  their  impacts.  Capturing  the  lessons  provide  opportunities  to  share  them  with 
others.  The  methodology  used  the  following  steps.  Other  case  studies  were  reviewed  to 
determine  if  previous  work  had  been  accomplished  and  if  there  was  merit  to  using  this 
approach.  The  review  confirmed  similar  case  studies  had  been  written  by  the  Air  Force 
and  National  Aeronautical  and  Space  Administration  (NASA)  (Chislaghi,  Dyer  and  Free 
2010;  Grenville,  Kleiner  and  Newcomb  2004;  Griffin  2004;  Griffin  and  Kinnu  2007; 
Jacques  and  Strouble  2010;  Kinzig  2010)  and  several  addressed  SOS  issues  (Collens  and 
Krause  2005;  Mattice  2003;  O’Brien  and  Griffin  2007).  The  Air  Force  recognized  the 
need  to  extract  lessons  learned  from  a  number  of  their  programs  after  acknowledging 
much  of  their  systems  engineering  expertise  had  atrophied.  Additionally,  NASA  faced  a 
similar  situation  when  it  recognized  that  its  workforce,  which  consists  of  highly 
specialized  and  experienced  engineers,  scientists  and  technicians,  had  a  significant 
percentage  approaching  retirement.  Capturing  their  knowledge  and  experience  in  order  to 
share  it  with  others  resulted  in  a  series  of  case  studies  analyzing  Air  Force  and  NASA 
programs. 
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Searches  for  Navy  and  specifically  command,  control,  computers, 
communications,  and  intelligence  (C4I)  case  studies  revealed  few  existed  for  C4I 
systems.  Further  searches  of  the  Program  Executive  Office  (PEO)  C4I  archives  had  none 
for  PEO  C4I  managed  systems.  Most  DOD  documentation  regarding  SOS  principles, 
characteristics,  requirements  and  acquisition  has  been  developed  only  recently.  The 
CSRR  program  documentation  provided  insight  to  the  history,  requirements  and  policies 
for  managing  the  CSRR  program.  Review  of  various  team  documents  and  interviews 
with  subject  matter  experts  from  the  engineering  and  production  teams  were  conducted  to 
capture  insight  about  developing  and  managing  a  SOS  program  and  the  challenges  of 
coordinating  with  the  constituent  systems.  The  compiled  information  was  then 
synthesized  to  identify  lessons  learned,  or  learning  principles,  develop  conclusions,  and 
make  recommendations  for  further  investigation. 

Developing  case  studies  meets  several  objectives.  Capturing  the  information 
about  a  particular  event,  person,  or  object  can  reveal  the  significant  issues  or  lessons 
learned.  These  lessons  learned  can  be  used  as  real  life  examples  to  train  engineers  and 
program  managers.  Application  of  these  lessons  can  aid  in  avoiding  repeating  mistakes, 
or  identify  similar  opportunities  to  improve  cost,  schedule  and  performance.  The  use  of 
case  studies  by  the  Navy  is  not  clearly  evident  but  the  Air  Eorce  and  NASA  have 
recognized  their  value  for  identifying  important  lessons.  Capturing  the  knowledge 
transfers  it  from  a  tacit  form  to  a  more  easily  accessible  explicit  format.  The  lack  of  case 
studies  about  C4I  systems,  particularly  those  managed  by  PEO  C4I,  identified  the  value 
of  examining  a  program  within  their  portfolio. 

CSRR  revealed  managing  a  SOS  program  has  a  number  of  challenges.  Most  DOD 
SOSs  are  classified  as  an  acknowledged  type  of  SOS  since  they  are  composed  of 
individual  programs  with  their  own  program  and  funding  responsibilities.  Acknowledged 
SOSs,  such  as  CSRR,  also  have  their  own  requirements  and  funding,  but  these  must  be 
synchronized  with  the  other  systems  within  the  CSRR  architecture.  Each  individual 
system  can  cause  emergence  to  other  systems  as  components  are  added  or  removed. 
Effective  governance  is  required  to  balance  the  requirements  of  the  constituent  systems 
composing  CSRR  within  the  SOS  architecture  (Vaneman  and  Jaskot  2013).  Changes  to 


any  of  the  constituent  systems  are  managed  by  their  respective  programs  but  must  be 
evaluated  by  the  overarching  SOS  to  avoid  or  minimize  degradation  or  disruption  of 
capability.  Attempting  to  optimize  one  system  over  the  others  can  be  detrimental  to  the 
overall  system  of  systems.  An  advantage  of  a  SOS  is  the  redundancy  not  available  from  a 
single  system  (Jamshidi  2009).  Disruption  of  a  communications  or  network  path  can  be 
mitigated  by  using  an  alternate  means. 

Systems  engineering  and  SOS  engineering  share  many  characteristics  but  differ  in 
their  approach  (Director,  Systems  and  Software  Engineering  2008).  A  system  engineer 
will  strive  to  develop  a  single  system  based  on  clearly  defined  requirements.  System  of 
systems  requirements  are  more  generalized  and  the  SOS  engineer  is  responsible  for 
integrating  the  capabilities  of  two  or  more  systems.  Systems  have  a  fairly  defined  life 
cycle.  System  of  systems  tend  to  be  more  perpetual.  The  various  life  cycles  typically  are 
not  aligned  so  a  system  of  systems  will  possess  an  evolutionary  life  cycle  which  changes 
but  does  not  end.  A  system  normally  has  a  single  program  manager  while  a  system  of 
systems,  depending  on  the  type,  may  not  have  one  at  all. 

Examination  of  CSRR  as  a  program,  process  and  product  provided  insight  to  the 
integration  approach  and  the  domains  related  to  requirements,  architecture,  design, 
integration  and  management.  As  a  program,  understanding  the  SOS  and  balancing  them 
with  the  constituent  systems  is  key  to  delivering  the  right  capabilities  to  the  user.  Erom  a 
product  consideration,  effective  management  of  interfaces  is  necessary  to  enabling  the 
right  capabilities  as  systems  are  integrated.  As  a  process,  implementing  SOS  engineering 
processes  acknowledges  emergence  may  occur.  All  of  these  have  a  bearing  on 
performing  successful  SOS  engineering. 

Common  Submarine  Radio  Room  is  the  culmination  of  these  efforts  while 
introducing  open  systems  architecture  designed  to  combine  and  leverage  its  constituent 
systems  to  deliver  capabilities  not  possible  in  an  individual  manner.  The  approach  for 
developing  CSRR  has  evolved  as  well,  moving  from  developing  a  specific  increment 
version  for  each  class  to  the  point  where  a  single  version  delivers  a  complete  core 
capability  capable  of  accounting  for  any  unique  platform  characteristics. 
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I.  INTRODUCTION  AND  BACKGROUND 


Today’s  submarine  communications  requirements  continue  to  increase  as 
bandwidth  and  network  capacity  expands.  Interoperability  challenges  between  the  various 
communications  systems  throughout  the  U.S.  military  place  additional  burdens  and 
vulnerabilities  on  the  warfighter.  While  there  are  specific  requirements  which  must  be 
addressed,  the  overall  capability  of  the  communications  systems  must  (a)  rapidly  adapt  to 
changing  demands  while  providing  the  right  information  to  the  right  place  at  the  right 
time,  (b)  protect  it  from  interception  and  exploitation,  and  (c)  deliver  it  in  a  format  which 
can  be  actionable  (Joint  Chiefs  of  Staff  [JCS]  2011).  Achieving  these  basic  objectives 
enable  U.S.  forces  to  accomplish  their  assigned  missions. 

The  challenge  of  meeting  the  demands  of  the  warfighter  has  forced  individual 
command,  control  and  communications  systems  to  integrate  more  closely  into  large  and 
complex  system  of  systems.  Many  systems  are  developed  without  consideration  of  how 
they  impact  other  systems  or  the  operations  and  support  infrastructure.  Common 
Submarine  Radio  Room  (CSRR)  provides  a  programmatic  path  to  oversee  integration 
activities,  an  architecture  supporting  coordinated  delivery  of  capabilities  and  physical 
products  to  achieve  interoperability  of  individual  systems  across  multiple  submarine 
platforms. 

The  CSRR  program  is  the  latest  effort  by  the  submarine  force  to  implement  a 
holistic  approach  to  design,  build,  certify,  and  deploy  a  command,  control, 
communications,  computers  and  intelligence  (C4I)  architecture  as  a  system  of  systems 
composed  of  individually  managed  programs  of  record  (FOR).  The  CSRR  program  was 
established  within  the  undersea  integration  program  management  warfare  office 
(PMW770)  to  assume  the  lead  systems  integrator  role  for  Program  Executive  Office  for 
Command,  Control,  Communications,  Computers  and  Intelligence  (PEO  C4I)  programs 
planned  for  deployment  to  a  submarine.  Using  a  robust  design,  build,  test  and  certify 
strategy  the  CSRR  has  successfully  demonstrated  it  is  operationally  effective  and 
suitable.  Since  the  initial  operating  capability  (IOC)  in  2006,  CSRR  is  fielded  on  all 

Virginia  (VA),  Ohio  ballistic  missile  (SSBN),  Ohio  guided  missile  (SSGN),  and  Seawolf 
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(SW)  platforms.  The  Los  Angeles  (LA)  class  began  in  2012  and  is  planned  to  reach  full 
operational  capability  (FOC)  in  fiscal  year  (FY)  2018.  Common  Submarine  Radio  Room 
as  a  product  architecture  and  program  strategy  has  demonstrated  the  effectiveness  of 
integrating  multiple  products  within  the  PEO  C4I  portfolio.  Since  the  original  increment 
one  version  zero  CSRR  has  continued  to  evolve.  Today  increment  one  version  three  (V3) 
is  being  fielded  to  LA  and  VA  platforms  with  SSBN,  SSGN,  and  SW  beginning  in  2014. 

The  success  of  the  CSRR  program  has  spurred  other  warfare  domains  to  examine 
how  the  CSRR  architecture  can  be  expanded  to  influence  other  platform  architecture  and 
engineering  strategies  and  create  a  product  line  for  the  other  communities  within  the  U.S. 
Navy.  In  order  to  capture  the  lessons  learned  from  the  CSRR  program  this  case  study  will 
analyze  the  history,  concept  of  operations,  systems  engineering,  the  results  and 
assessment  of  the  benefits  and  costs.  The  lessons  learned  from  the  CSRR  program  as  a 
system  of  systems  engineering  and  integration  activity  can  be  identified  and  passed  onto 
other  programs.  Acknowledgment  by  PEO  C4I  the  CSRR  model  works  serves  as  a  key 
testament  to  the  viability  of  using  a  system  of  systems  design  approach.  Today  PEO  C4I 
is  evaluating  the  idea  of  a  “Common  Radio  Room”  for  surface  combatants. 

A,  WHY  A  CASE  STUDY? 

Case  studies  provide  the  opportunity  to  capture  and  distribute  valuable  lessons 
learned.  Case  study  approaches  can  vary  in  their  format  and  goals.  Case  studies  are  used 
to  perform  the  following  (National  Aeronautics  and  Space  Administration  [NASA] 
Goddard  Space  Elight  Center  [GSEC]  2011;  Haskins  2012): 

•  Record  mission  or  project  successes  or  failures 

•  Lessons  learned  of  a  technical  or  programmatic  nature 

•  Design  decisions  of  what  worked  or  did  not  work  and  the  outcomes 

•  Incidents,  near  incidents  and  safety  reminders 

•  Personal  insights 

Eriedman  and  Sage  (2003,  84-96)  discuss  how  case  studies  can  contribute  to 
capturing  the  history  of  a  program  or  event  for  systems  engineering,  systems 
management  and  acquisition.  Case  studies  support  teaching  students  about  problems 
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experienced  in  the  real  world.  Effective  case  studies  capture  lessons  learned  during  the 
different  phases  of  the  program  life  cycle  so  they  can  be  shared  with  others.  Sharing  in 
turn  provide  insight  for  future  systems  engineers  and  program  managers  tasked  with 
developing  and  managing  a  system  of  systems  program  to  understand  the  challenges  and 
opportunities  and  how  to  avoid,  minimize,  or  leverage  them.  The  use  of  a  case  study 
framework  provides  an  effective  means  to  decompose  the  issues  into  a  specific  topic  and 
responsibility. 

B,  PURPOSE  OF  THIS  CASE  STUDY 

Research  both  online  and  available  libraries  identified  there  are  few  case  studies 
concerning  the  application  of  system  of  systems  (SOS).  Many  systems  case  studies  exist 
but  the  concept  of  a  SOS  has  only  been  widely  acknowledged  recently.  This  thesis  will 
examine  the  CSRR  program  and  attempt  to  provide  lessons  that  can  be  applied  to  other 
SOS  in  terms  of  the  following: 

1.  The  history  of  submarine  communications  leading  up  to  the  CSRR 
program. 

2.  The  organizational  structure  of  the  CSRR  program. 

3.  The  relationship  with  other  programs  of  record  and  stakeholders. 

4.  SOS  architecture  management. 

5.  The  advantages  and  disadvantages  of  the  CSRR  SOS  approach  within  the 
various  disciplines  (e.g.,  development,  modernization,  integrated  logistics 
support  (ILS),  training,  sustainment,  and  information  assurance  (lA)). 

6.  Process  improvement  initiatives  and  their  impact  in  regard  to  cost, 
schedule  and  performance. 

7.  CSRR’s  ability  to  meet  future  mission  requirements  while  supporting 
current  missions. 

C.  RESEARCH  QUESTIONS 

Research  into  the  development  of  CSRR  and  management  of  a  SOS  identified 
several  questions.  This  thesis  will  examine  the  following  questions  to  provide  a  clearer 
understanding  of  how  PEO  C4I  develops  their  individual  programs  and  integrates  them 
into  a  larger  system  of  systems  program  such  as  CSRR. 

1 .  What  is  CSRR  and  what  characteristics  classify  it  as  a  system  of  systems? 
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2.  What  are  the  benefits  and  ehallenges  of  developing,  designing,  produeing, 
deploying,  and  sustaining  CSRR  as  a  SOS? 

3.  What  best  praetiees  have  been  identified  and  implemented  in  the  CSRR 
program  and  what  benefits  have  been  realized  in  terms  of  cost, 
performance,  and  schedule? 

4.  What  lessons  learned  can  be  applied  to  future  versions  of  CSRR  and  CRR 
for  surface  combatants? 

D,  SCOPE 

This  assessment  will  look  at  the  CSRR  program  from  the  following  perspective: 

1 .  The  development  of  submarine  communications  from  its  initial  beginnings 
up  through  the  deployment  of  CSRR  increment  one  version  three. 

2.  The  organizational  structure  of  the  CSRR  program  to  include  the  design 
and  development  group,  production  and  installation  group,  ITS  and 
training  groups,  lA  groups,  and  sustainment  group. 

3.  The  version  development  process,  its  strengths  and  weaknesses. 

4.  SOS  architecture  management  with  other  programs  of  record  and  portfolio 
capability  management,  the  relationships  with  the  other  programs  of 
record  and  the  warfighter. 

5.  The  advantages  and  disadvantages  of  the  CSRR  system  of  systems 
approach  regarding  ITS,  training,  production,  installation  (synchronization 
of  installations  into  block  upgrades),  lA  and  sustainment. 

6.  Assessment  of  requirements  in  a  changing  environment  with  regard  to  the 
Undersea  Connectivity  Roadmap,  Design  for  Undersea  Warfare,  PEO  C4I 
Master  Plan,  and  the  way  ahead  for  considering  disruptive  technologies. 

7.  An  evaluation  of  the  process  improvement  initiatives  and  their  influence 
and  impact  in  regard  to  cost,  schedule  and  performance. 

8.  The  future  of  CSRR  in  today’s  environment  and  tomorrow  up  through 
2030. 

E.  METHODOLOGY 

The  methodology  used  in  this  case  study  consisted  of  the  following  activities. 

1 .  Investigation  into  other  case  studies  to  determine  if  other  researchers  had 
performed  similar  work  and  confirm  if  a  case  study  would  be  an 
appropriate  approach.  Review  of  other  case  studies  did  indicate  similar 
work  had  been  done  but  no  specific  case  studies  had  been  found 
specifically  addressing  specific  programs  as  a  SOS. 

2.  Investigation  into  Navy  and  specifically  PEO  C4I  archives  to  determine  if 
any  case  studies  had  been  written. 
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3.  Review  of  the  DOD  aequisition  and  program  documentation  regarding 
SOS,  defense  acquisition  requirements,  systems  and  system  of  systems 
principles. 

4.  Perform  an  in  depth  analysis  of  the  CSRR  program  documentation.  This 
includes  the  formal  program  documentation  and  minutes  from  the  various 
integrated  product  teams  (IPX)  supporting  the  program. 

5.  Conducted  selected  interviews  with  subject  matter  experts  (SME)  with 
regard  to  developing  and  managing  a  SOS  program  and  the  individual 
systems  supporting  the  SOS. 

6.  Synthesize  the  information  to  capture  lessons  learned  (or  learning 
principles),  develop  conclusions  and  make  recommendations  for  further 
consideration.  A  derivative  of  the  Friedman  and  Sage  framework  will  be 
used  since  contractor  involvement  is  limited. 

F.  EXPECTED  BENEFITS  OF  THIS  CASE  STUDY 

Warfighter  capabilities  increasingly  involve  using  complex,  disparate,  and 
geographically  separate  systems.  PEO  C4I  manages  over  100  programs  and  many  more 
projects  and  investigations.  The  importance  of  programs  is  determined  by  the  acquisition 
category  (ACAT)  assigned  which  is  primarily  related  to  the  expected  program  cost  and 
not  its  complexity  (Carter  2013,  Enel  3).  Every  program  manager  wants  to  be  successful 
in  managing  the  activities  and  funding  aligned  to  their  program.  Management  of 
individual  programs  versus  management  by  capability  creates  unexpected  issues  and 
friction  as  systems  are  deployed  in  different  environments.  Many  programs  fail  to 
develop  synergy  with  others  to  improve  or  expand  their  capabilities.  Systems 
requirements  and  system  of  systems  requirements  often  conflict  forcing  unexpected 
changes.  For  example  an  individual  system  may  need  to  meet  a  higher  availability 
requirement  in  order  to  help  the  SOS  meet  its  availability  threshold  requirement. 

The  benefit  of  examining  how  the  CSRR  program  is  managed,  the  processes  it 
developed,  relationships  with  stakeholders,  and  its  successes  and  failures  can  provide  a 
guide  for  managing  a  complex  SOS  from  development  through  evolution/modernization 
and  sustainment. 
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II.  LITERATURE  REVIEW 


There  is  a  great  deal  of  literature  available  about  systems,  systems  engineering, 
systems  of  systems  engineering  and  the  development  and  use  of  case  studies.  Available 
case  studies  of  other  government  programs  identify  learning  principles  of  the  positive  and 
negative  aspects.  A  majority  of  CSRR  program  information  is  normally  limited  to  what  is 
developed  within  the  program  office  to  support  its  acquisition  responsibilities.  This 
includes  the  requirements  documentation,  engineering  plans,  test  and  evaluation  plans, 
acquisition  strategies,  concept  of  operations,  etc.  Understanding  the  activities  and  events 
which  led  up  to  the  development  of  CSRR  look  at  how  communications  evolved  from 
simple  single  function  components  to  complex,  multi-functional  voice  and  information 
network  nodes.  These  documents  are  important  as  each  supports  the  systems  engineering 
activities  necessary  for  developing  and  managing  an  acquisition  program. 

This  chapter  will  look  at: 

1 .  The  background  of  submarine  communications  leading  up  to  CSRR 

2.  Available  CSRR  program  documentation  to  include  acquisition, 
engineering,  test  and  evaluation,  logistics  and  information  assurance  (lA) 

3.  Available  systems,  systems  engineering  and  system  of  systems 
engineering  documentation 

4.  Systems  engineering  case  studies 

A  search  of  the  Dudley  Knox  Library  for  the  term  “Navy  system  engineering  case 
studies”  identified  96  possible  candidates.  Performing  a  more  detailed  search  for  a 
‘“system  of  systems’”  Navy  engineering  case  study”  identified  only  five  hits.  Using 
Google  to  perform  a  similar  search  generated  over  25,000,000  possible  results.  Extending 
this  further  for  the  term  “system  of  systems”  narrowed  the  results  to  over  350,000.  In 
each  search,  several  of  the  example  case  studies  were  listed.  The  search  for  case  studies 
identified  those  written  by  the  Air  Force  and  NASA.  The  Hubble  space  telescope 
(Mattice  2003),  F-111  (Richey  2005),  Global  Hawk  (Kinzig  2010),  Theater  Battle 
Management  Core  System  (TBMCS)  (Collens  and  Krause  2005),  C-5  Galaxy  (Griffin 
2004)  and  several  others  were  used  in  support  of  the  research. 
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A. 


SUBMARINE  COMMUNICATIONS  BACKGROUND 


The  modem  submarine  foree  has  been  in  existenee  since  1900.  Since  the 
beginning  development  of  effective  and  reliable  technology  capable  of  supporting 
submarine  command  and  control  (C2)  has  always  proved  to  be  challenging.  The 
invention  of  wireless  telegraphy  provided  the  ability  to  communicate  between  a 
submarine  and  another  location.  The  U.S.  Navy  began  experimenting  with  submarine 
communications  in  1912  successfully  testing  the  capability  at  a  range  of  four  nautical 
miles  off  Newport,  Rhode  Island  (Howeth  1963,  513-546).  Following  World  War  One  a 
100  watt  submarine  transmitter,  a  model  TM,  provided  the  initial  capability  of  a  spark 
gap  radio.  At  the  same  time,  the  Navy  teamed  with  the  Edison  Society,  Bureau  of 
Standards,  Hammond  Laboratory  and  Marconi  Telegraph  to  determine  how  to 
communicate  with  submarines.  Demonstrations  of  different  antennas  and  radios  resulted 
in  a  combination  capable  of  receiving  very  low  frequency  (VLF)  signals  from  distances 
up  to  3,000  miles  (Howeth  1963,  319-335)  while  submerged  at  periscope  depth.  The 
1920s  saw  the  invention  and  expansion  of  high  frequency  (HE)  communications  with  the 
Navy  successfully  demonstrating  the  technology  to  reliably  communicate  from  ship  to 
ship  and  ship  to  shore  via  voice.  Navy  leadership  recognized  their  growing  submarine 
force  needed  a  reliable  means  to  receive  long  range  communications  in  all  planned  areas 
of  operation.  Another  unintended  aspect  during  the  1920s  was  the  increase  of  commercial 
radio,  which  caused  a  clash  between  military  and  civil  interests.  The  competition  for 
access  to  the  frequency  spectrum  drove  investigation  into  using  higher  frequencies 
expanding  into  the  very  high  frequency  (VHF)  and  ultra-  high  frequency  (UHF)  spectrum 
(Howeth  1963,  397-402). 

Communications  continued  to  play  a  key  role  during  World  War  Two.  Vice 
Admiral  Thomas  Hart,  commander  of  the  Asiatic  Fleet,  used  low  frequency  (LF)  radio  to 
initiate  unrestricted  submarine  warfare  following  the  attack  on  Pearl  Harbor.  At  the  time 
high  frequency  was  acknowledged  as  the  primary  long  haul  communications  path. 
Unfortunately,  HF  suffered  from  interference  caused  by  weather,  sun  spots,  diurnal 
effects,  and  could  not  be  utilized  while  submerged.  U.S.  submarines  copied  the  German 
model  of  wolf  packs,  where  submarines  coordinated  their  operations,  which  in  turn 
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identified  a  need  for  short  range  communieations.  Additional  threats  from  attaeks  by 
friendly  aireraft  emphasized  the  need  as  well.  The  deployment  of  VHP  radios  addressed 
this  need  near  the  end  of  the  war  and  the  post  war  period  (Clay  2008).  Figure  1  illustrates 
the  typical  radio  room  of  a  WWII  fleet  submarine. 

The  Cold  War  and  nuclear  arms  race  with  the  Soviet  Union  created  requirements 
for  greater  C2  and  weapons  systems  capabilities,  driving  further  advances  in  submarine 
communications.  However,  all  systems  up  to  this  point  were  still  a  single  function, 
stovepipe  capability.  The  expansion  of  the  VLF  usage  and  establishment  of  transmitters 
capable  of  a  global  reach  provided  one  reliable  path  for  supporting  communications  with 
submarines.  Additional  communications  circuits  were  added  as  the  submarine  missions 
expanded  and  additional  radio  frequency  (RF)  spectrum  became  available.  The  USS 
Nautilus  radio  room  shown  in  Figure  2  is  more  modular  but  still  maintains  unique 
functions  within  a  specific  box. 


Figure  1 .  WW2  USS  Torsk  Radio  Room  (from  Hummel  n.d.) 
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Figure  2.  Replica  of  USS  Nautilus  Radio  Room  (from  Amateur  Radio  Relay 

League  2009) 


Deployment  of  the  SSBN  USS  George  Washington  in  late  1960  (Yamell  n.d.), 
shown  in  Figure  3  was  supported  with  the  capabilities  of  the  Fixed  Submarine  Broadcast 
System  (FSBS)  in  the  VLF  spectrum  and  establishment  of  transmitters  capable  of  a 
global  reach  to  provide  reliable  one  way  continuous  communications  with  the  National 
Command  Authority  (NCA).  The  VLF  Digital  Information  Network  (VERDIN)  served  as 
the  shipboard  system  of  the  FSBS  to  receive  and  process  messages  to  the  message 
processor.  The  VERDIN  systems  were  phased  out  in  the  late  1990s  to  be  replaced  with 
the  Submarine  LE/VEE  Versa  Modular  Eurobus  (VME)  Receiver  (SEVR). 
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Figure  3.  USS  George  Washington  SSBN-598  (from  Yarnell  n.d.) 


The  1960s  ushered  in  the  era  of  global  satellite  communications  as  the 
commercial  industry  realized  the  potential  of  using  satellites.  NASA  launched  the  first 
Telstar  satellite  in  1962  and  Syncom  three,  shown  in  Figure  4,  became  the  first 
geosynchronous  satellite  providing  television  coverage  of  the  Olympics  in  Tokyo  (King 
and  Ricchio  2010). 

Successful  launches  in  the  1960s  and  1970s  of  communications  satellites  added 
UHF  capability  with  the  Submarine  Satellite  Information  Exchange  System  (SSIXS) 
(Chief  of  Naval  Operations  [CNO]  N61  and  N87  1998,  B-15).  Unlike  VLF 
communications  which  had  a  slow  data  rate  SSIXS  provided  a  near  real  time  means  for 
tactical  and  strategic  communications.  Additional  communications  circuits  were 
developed  by  leveraging  derivatives  of  established  data  management  architectures,  such 
as  the  Battle  Group  Information  Exchange  Subsystem  (BGIXS)  and  Officer  in  Tactical 
Command  Information  Exchange  System  (OTCIXS)  to  provide  direct,  bi-directional 
communications  between  battle  group  units  and  a  submarine  (CNO  N61  and  N87  1998, 
A-34,  B-15;  Naval  Networks  Warfare  Command  [NETWARCOM]  2008). 
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Figure  4.  SYNCOM  Satellite  (from  NASA  2009) 


These  systems  were  operational  well  into  the  first  deeade  of  the  twenty-first 
century  before  being  retired.  Extremely  high  frequency  (EHF)  capable  radios  were 
introduced  in  the  1970s  with  the  Defense  Satellite  Communications  System  (DSCS)  and 
expanded  to  include  a  larger  segment  of  the  RE  spectrum.  The  Military  Strategic  and 
Tactical  Relay  System  (MILSTAR)  EHF  system  demonstrated  the  capability  to  provide 
protected  communications  in  a  contested  environment  (King  and  Ricchio  2010). 

Several  classes  of  submarines  entered  service  along  with  the  SSBN.  The  USS 
Permit  class  entered  service  in  the  early  1960s  followed  rapidly  by  the  USS  Sturgeon 
class.  The  USS  Los  Angeles  (LA)  class  submarines  entered  service  in  1976  as  the 
replacement  for  the  USS  Sturgeon  class.  Originally  conceived  to  be  a  member  of  a  carrier 
strike  group  or  battle  group  there  was  a  greater  emphasis  on  using  satellite  and  LOS 
communications  circuits.  The  communications  capabilities  for  all  of  these  classes  were 
similar  in  terms  of  systems  being  installed  and  operated  in  a  stovepipe  fashion. 
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The  USS  Ohio  SSBN  Trident  submarines  were  the  first  platforms  to  deliver  an 
integrated  eommunieations  eapability.  The  integrated  radio  room  (IRR)  was  a 
eommereially  provided  solution  based  on  1970s  teehnology  delivered  by  the  shipbuilder, 
Eleetric  Boat.  Heavily  oriented  toward  reliable  eommunieations  links  the  IRR  was  a 
eontraetor  delivered  system  speeifically  built  to  support  the  submarine  strategic  mission. 
Designed  with  a  high  degree  of  automation,  the  IRR  was  centrally  operated  by  several 
watch  standers  responsible  to  ensure  continuous  communications  in  order  to  act  on  orders 
from  the  NCA  received  via  emergency  action  messages  (EAM).  The  engineering 
approach  proved  the  IRR  design  was  robust  but  its  proprietary  design  proved  to  be  too 
expensive  to  maintain  and  modernize  (NUWC  2008,  21).  Some  minor  standalone 
changes  were  accomplished  in  the  1990s  to  meet  the  changing  technology  of  Internet 
Protocol  (IP).  The  last  IRR  was  removed  from  service  in  2011  with  the  installation  of 
CSRR  increment  one  version  one  (VI). 

The  tactical  communications  system  was  conceived  in  the  1980s  as  an  attempt  to 
leverage  the  Trident  centralized  control  capability  to  improve  nuclear  attack  submarine 
(SSN)  radio  room  operability.  SSN  communications  circuits  at  that  time  required  many 
steps  to  lineup,  providing  opportunities  for  operator  error  (NUWC  2008,  15).  The  USS 
Seawolf  (SW)  class,  designed  as  follow  on  to  the  EA,  was  planned  to  use  a  commercially 
furnished  equipment  design  similar  to  the  approach  of  the  Trident  but  also  planned  to 
introduce  centralized  RE  and  baseband  switching.  Designed  primarily  as  a  Cold  War 
response  to  the  Soviet  Navy  the  significant  procurement  cost  and  the  end  of  the  Cold  War 
limited  the  SW  procurements  to  three  platforms.  Since  then  they  have  been  modernized 
with  CSRR.  The  demise  of  the  SW  program  drove  the  development  of  another 
replacement  for  the  LA  class.  The  new  SSN  program  began  development  in  the  early 
1990s  which  ultimately  became  known  as  the  USS  Virginia. 

In  the  early  1990s  the  Navy  acknowledged  an  integrated  communication  solution 
was  needed.  Mission  needs  statement  (MNS)  M063 -06-95  established  the  need  for  an 
Integrated  Maritime  Communications  System  (IMCS)  in  support  of  maritime  and  joint 
C4I  (CNO  N8I  1995).  The  MNS  outlined  the  IMCS  requirements  to  encourage  improved 
reliability,  survivability,  standardization,  flexibility,  data  formats  and  throughput,  use  of 
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commercial-off-the-shelf  (COTS),  government-off-the-shelf  (GOTS)  and  non- 
developmental  item  (NDI)  components,  provide  multi-level  seeurity,  and  reduce  life 
eyele  costs.  Table  1  lists  the  objeetives  for  the  IMCS  and  Table  2  outlines  the  general 
eapabilities  from  the  MNS. 


Table  1.  IMCS  MNS  Objectives  (after  CNO  N81  1995) 


Integrated  Maritime  Communications  System  Objectives 

(I)  Improved  shipboard  information  transfer  eapability  by: 

(a) 

Providing  reliable  and  survivable  communieations  conneetivity,  inereased 
variable  information  transfer  capaeity,  and  timely  dissemination  in  a  stressed 
environment 

(b) 

Providing  forees  with  flexibility  to  rapidly  re-align  communieations  service 
in  response  to  changing  operational  needs 

(c) 

Providing  forees  with  the  new  information  transfer  technologies 
encompassed  in  personal  eommunications  serviees 

(2)  Implement  improved  information  transfer  capabilities  through  evolutionary  and 
incremental  phasing  by: 

(a) 

Standardization  of  hardware,  algorithms,  data  formats  an  operational 
procedures 

(b) 

Use  of  dynamic  reprogrammable  architecture. 

(c) 

Use  of  open  system  arehitecture  to  ensure  delivery  of  all  source  SCI 
information  and  data 

(d) 

Introducing  new  Non-Developmental  Item  (NDI)  antenna  system,  (e.g., 
Pico-eells)  to  improve  survivability  and  provide  user  location 

(3)  Introd 

uce  state-of-the-art  teehnology  into  the  information  transfer  process  by: 

(a) 

Shared  use  of  equipment  with  parallel/redundant  capacity  and  RF  links. 

(b) 

Maximizing  efficiency  of  resources  aceess  eontrol  and  sub-network 
processing  algorithms 

(c) 

Multimedia  networking 

(d) 

Automation  of  system  eontrol,  monitoring,  setup,  and  information 
dissemination 

(e) 

Full  use  of  non-developmental  items  (NDI),  commercial-off-the-shelf 
(COTS),  and  government-off-the-shelf  (GOTS)  applications 

(4)  Reduce  or  eliminate  the  dependence  on  tethered  communications  devices  without 
hindering  the  continued  need  to  disseminate  taetical  data  in  near  real  time  within  the 
fleet  and  from  shore  to  the  fleet 
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Table  2.  Integrated  Maritime  Communieations  System  General  Capabilities 

(fromCNONSl  1995) 


IMCS  General  Capabilities 

1 

Decant  and  direct  operational  data  onto  information  pathways  where  operational 
priorities  can  be  set  and  managed  by  doctrine  established  to  manage  the  system  with 
minimal  operator  intervention 

2 

Rapidly  reconstitute  essential  capabilities  during  degraded  modes  of  operation 
while  maintaining  continuity  of  information.  This  reconstitution  may  be 
accomplished  by  the  use  of  redundant  or  reconfigurable  equipment 

3 

Employ  data  compression,  object-oriented  transmission  packets,  “delta” 
transmission  (e.g.,  sending  only  the  part  of  data  files  that  actually  change  between 
transmission) 

4 

Accommodate  dramatic  change  in  information  format  characterized  generally  by 
voice,  video  teleconferencing,  imaging  and  digital  data,  not  principally  character- 
oriented  textual  information 

5 

Employ  more  efficient  formats,  predominantly  using  binary  data  files,  displayed  as 
high  resolution  graphics 

6 

Provide  full  media  capability 

7 

Perform  dynamic  bandwidth  management  using  full  parallel/redundant  networks 

8 

Provide  multi-level  security 

9 

Provide  reduced  life  cycle  costs 

The  submarine  communications  support  system  (SCSS)  was  developed  in  the 
1990s  in  response  to  the  IMCS  MNS  (CNO  N81  1995)  and  the  release  of  the  original 
Submarine  Communications  Master  Plan  (SCMP)  (CNO  N87  1995).  The  SCSS  began  an 
incremental  approach  to  modernizing  the  submarine  radio  room.  The  FYOO  revision  to 
the  SCMP  (CNO  N61  and  N87  1998,  ii-iii)  augmented  the  MNS  requirements  as  well  as 
defining  the  phases  for  the  SCSS  with  the  following; 

The  SCSS  must  be  a  cost-effective  system  architecture,  with  emphasis  on 
maximizing  commonality  between  the  SCSS  suites  on  all  classes  of 
submarines.  This  comprehensive  development  and  installation  plan 
integrates  all  current  communications  improvement  programs  in  a  time 
phased  implementation,  taking  into  account  the  rapid  development  of 
communications  technology.  The  defined  phases  (Automated  Message 
Handling  phase  (FY94-96),  Automated  Signal  Routing  phase  (FY97-98), 
Automated  Radio  Room  phase  (FY99-05))  will  transition  the  current 
radio  rooms  to  a  hybrid  SCSS,  based  largely  on  modem,  open  systems 
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architecture  (OSA)  radios  and  switching  equipment,  with  some  legacy 
equipment  retained. 

The  SCSS  used  the  submarine  message  buffer  to  provide  the  automated  message 
handling  while  the  submarine  baseband  circuit  switch  (SBCS  or  BBS)  and  miniature 
demand  assigned  multiple  access  (MINI-DAM A),  shown  in  the  land  based  submarine 
radio  room  (LBSRR)  in  Figure  5,  provided  the  automated  baseband,  RF  switching  and 
improved  UHF  signal  routing  using  COTS  and  NDI  solutions.  Each  of  these  components 
was  still  an  individual  system  within  the  block  upgrade  approach  which  packaged 
capabilities  and  implemented  them  within  the  wideband  modernization  plan.  Packaging 
these  systems  enabled  them  to  be  integrated,  tested  and  installed  as  a  complete  set  of 
capabilities  (NUWC  2008). 


Figure  5.  Submarine  Communications  Support  System  Pre -baseband  Switch  in 
the  Land  Based  Submarine  Radio  Room,  NUWC  Newport  (from 

Keller  2012) 
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The  SCSS  was  not  a  formal  program,  but  a  concept  proving  individual  component 
programs  could  be  integrated  in  order  to  deliver  greater  C4I  capability.  Thus  SCSS 
became  the  first  generation  to  demonstrate  the  integration  of  communications, 
networking,  and  automation  could  meet  the  MNS  objective  and  capabilities  but  was 
limited  to  the  LA  class  submarines.  An  online  article  written  by  the  engineers  working 
for  the  Naval  Undersea  Warfare  Center  (NUWC)  Newport  and  posted  by  the  submarine 
warfare  directorate  described  SCSS  in  the  following  quote 

A  key  element  in  this  new  architecture  is  the  Submarine  Communications 
Support  System  (SCSS),  which  adapts  Navy-wide  communications 
components  and  capabilities,  while  minimizing  dependence  on  submarine- 
unique  equipment.  The  SCSS  will  use  industry-standard  protocols  and 
commercial  technology  in  hardware  ruggedized  for  the  rigors  of  the 
shipboard  environment.  Its  architecture  will  phase  out  today’s  “stovepipe” 
systems  to  implement  a  client-server  environment  for  exchanging 
information  by  means  of  seamless  and  comprehensive  connectivity  on 
shared,  common-user  communication  links.  (Longacre,  Exley  and 
Macmillan  1998) 

These  early  radio  suites  provided  a  broad  spectrum  of  communications  capability 
and  some  automation  but  their  stove  piped  programmatic  and  technical  approaches 
limited  the  potential  of  a  more  effective  and  robust  C4I  system.  SCSS  and  IRR  consisted 
mostly  of  components  unique  to  the  submarine  but  there  was  little  commonality  between 
their  architectures.  Sailors  transferring  from  a  SSN  to  a  SSBN  required  extensive 
retraining  prior  to  reporting.  Even  within  the  classes  platform  configurations  would  vary 
greatly.  Eack  of  configuration  management  and  control  led  to  sailors  developing  their 
own  operating  and  technical  documentation  to  operate  and  maintain  their  radio  rooms. 
IRR  maintained  tight  configuration  control  but  at  the  expense  of  not  maintaining  pace 
with  technology  changes  in  the  overall  military  communications  architectures. 

The  Submarine  Exterior  Communications  System  (SUBECS)  Capstone 
Requirements  Document  (CRD)  01-87-98  (CNO  N8  1998)  provided  the  specific 
requirements  for  the  SUBECS.  The  CRD  serves  as  the  operational  requirements 
document  (ORD)  since  the  joint  capabilities  integration  development  system  (JCIDS) 
process  was  not  yet  in  existence.  The  CRD  also  highlighted  the  limitations  of  the  current 

submarine  C4I  systems.  Stovepipe  systems,  limited  throughput,  manual  operations,  and 
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limited  data  storage  were  several  of  the  signilieant  areas  of  interest.  The  CRD  deseribed 
the  operational  eapability  of  the  SUBECS  to  provide  attaek  and  fleet  ballistie  missile 
submarines  with  seeure,  reliable,  eovert  eommunications,  and  effectively  manage, 
control,  process,  and  disseminate  Command,  Control,  Communications,  Computers  and 
Intelligence  (C4I)  information  (CNO  N8  1998). 

The  SUBECS  also  had  to  be  interoperable  with  the  global  command  and  control 
system  maritime/  defense  information  infrastructure-common  operating  environment 
(GCCS-M/DII-COE),  joint  maritime  command  information  system  (JMCIS),  and  joint 
maritime  communications  system  (JMCOMS).  Eurthermore,  the  CRD  stipulated  the  new 
system  must  use  an  open  systems  architecture  approach  while  still  meeting  the 
interoperability  requirements.  The  systems  supporting  the  SUBECS  (e.g.,  SEVR,  MINI- 
DAMA,  and  BBS)  have  their  own  requirements  documentation  defining  key  performance 
parameters  for  frequency  coverage,  information  routing  efficiency,  aggregate  system 
throughput,  and  operational  availability  (Ao). 

Revision  one  to  the  CRD  (CNO  N8  2003)  mandated  the  SUBECS  will  not 
develop  unique  C4I  solutions  while  adding  requirements  for  interoperability  with  the 
joint  technical  architecture  standards,  joint  tactical  radio  system  (JTRS),  Navy  Marine 
Corps  Intranet  and  naval  integrated  information  network.  SUBECS  planned  to  use  a 
spiral  development  approach  as  technology  evolved  and  became  available.  As  a  system 
of  systems,  the  SUBECS  leverages  other  capstone  requirements  documents  to  achieve 
compliance. 

The  CRD  provided  the  requirements  for  the  Virginia  (VA)  SUBECS.  The 
EY2000  Director  of  Operational  Test  and  Evaluation  (DOT&E)  report  described  the 
SUBECS  as  “an  umbrella  program,  which  integrates  fifteen  smaller  acquisition  programs 
and  commercial  off-the-shelf  (COTS)  components  into  a  system  that  supports  network 
centric  warfare”  (DOT&E  2000,  section  IV-167).  PMS450,  the  VA  program  office, 
originally  envisioned  implementing  an  unmanned  radio  suite  capable  of  automatically 
managing  communications.  The  manned  capability  was  reinstated  at  the  request  of 
Commander  Submarine  Eorces  (COMSUBEOR)  due  to  the  observed  technology 
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limitations.  The  modified  version  of  the  SCSS  design  for  the  LA  elass  served  as  the 
initial  plan  for  the  VA. 

In  the  late  1990s,  asymmetric  communications  using  submarine  Internet  Protocol 
(IP)  began  fielding  as  a  replacement  to  the  legacy  circuits  such  as  SSIXS  and  OTICXS. 
The  deployment  of  submarine  IP  proved  to  be  a  complex  and  lengthy  endeavor  since 
engineers  had  to  devise  solutions  to  integrate  contemporary  and  legacy  systems.  Initially 
planned  as  a  two  year  effort  in  1998  the  full  deployment  of  IP  capability  to  all  platforms 
did  not  reach  FOC  until  2007.  Even  then  there  were  unique  solutions  for  each  submarine 
class.  However,  the  solutions  did  reflect  the  initial  development  of  a  larger  overall  open 
systems  architecture. 

Commanding  officers  guidance  in  revision  one  of  the  Design  for  Undersea 
Warfare  (DUSW)  (Richardson,  Caldwell  and  Breckenridge  2012)  emphasizes  the 
capability  to  rapidly  shift  postures  from  complete  communications  silence  to  being  fully 
engaged  with  other  Navy,  DOD,  or  other  government  agencies  to  provide  support  as 
needed.  The  DUSW  emphasizes  a  high  level,  broad  requirement  for  providing  systems 
capable  of  operating  with  unmanned  aerial  or  undersea  vehicles  in  a  myriad  of 
environments.  The  commander’s  guidance  lists  communications  as  a  key  area  of 
proficiency.  The  DUSW  provides  additional  support  to  the  currently  defined 
requirements  for  developing  an  effective  SOS  capable  of  meeting  the  requirements  for 
the  submarine  warfighter. 

The  advances  of  network  technology  and  increasing  use  of  COTS  greatly 
increased  the  complexity  of  delivering  submarine  C4I  capabilities.  These  complexities 
have  proven  to  be  a  challenge  for  acquisition,  engineering,  and  logistics.  In  many  cases 
the  legacy  systems  had  reached  their  maximum  capabilities  and  were  approaching  end  of 
life.  The  next  step  to  address  these  challenges  required  considering  a  new  approach. 
Common  Submarine  Radio  Room  was  the  outcome. 

B,  COMMON  SUBMARINE  RADIO  ROOM  DOCUMENTATION 

PMW770  maintains  the  documentation  to  support  the  CSRR  program.  Using  the 
DOD  5000  series  acquisition  instructions  and  memorandums,  the  required  documentation 

19 


is  created  and  updated  as  neeessary.  Some  documents  such  requirements  documents, 
aequisition  plans  and  strategies  remain  fairly  static  once  signed.  Others  sueh  as  the  test 
and  evaluation  master  plan  and  system  engineering  plan  are  updated  as  necessary  to 
support  key  program  events.  The  SCMP  to  support  the  FY2000  program  objeetive 
memorandum  (CNO  N61  and  N87  1998)  is  an  update  to  the  original  SCMP  drafted  in 
1995.  It  provides  a  complement  to  the  SUBECS  CRD  rev  one  (CNO  N8  2003)  as  well  as 
serving  as  a  compendium  of  related  aequisition  requirements.  Proeess  documents  have 
been  developed  to  aid  in  capturing  the  processes  for  design,  development  and  testing,  and 
aequisition  planning.  Information  assuranee  (lA),  or  Cyberseeurity,  drives  a  whole  series 
of  doeuments  whieh  captures  the  relationship  of  the  components  of  the  CSRR. 

1,  What  is  Common  Submarine  Radio  Room 

CSRR  is  a  network-centrie  communieations  system  designed  to  support 
submarine  force  C4I  requirements.  CSRR  was  designed  to  provide  seamless,  transparent, 
secure  conneetivity  for  information  exchange  between  submarines  and  other  joint,  naval, 
DOD,  federal,  allied  and  coalition  force  (PMW770  2008,  9).  Figure  6  is  the  operational 
view  (OV)  one  (OV-1)  from  the  CSRR  capability  production  document  (CPD)  (PMW770 
2006). 

Originally  an  ACAT  III  program  in  2001  (PMW  173  2002,  5),  CSRR  beeame  the 
next  step  towards  a  common,  modular  open  systems  arehitecture  while  expanding 
automating  communications  network  management.  The  initial  CSRR  was  based  on  a  VA 
elass  submarine  contraetor  furnished  design.  CSRR  was  reclassified  as  an  ACAT  II 
program  in  2005  (ASN  (RDA)  2005)  based  on  the  revision  to  the  DOD  aequisition 
guidance  whieh  updated  the  funding  levels  for  development  and  procurement. 

After  seeing  signifieant  eost  increases  in  the  delivery  of  the  first  VA  CSRR, 
Program  Executive  Officer  Submarines  (PEO  SUB)  directed  an  analysis  of  alternatives 
(AOA)  be  performed  to  determine  the  optimal  aequisition  strategy  to  use.  The  AOA 
resulted  in  the  program  responsibilities  for  the  lead  systems  integrator  being  managed  by 
the  government  while  the  original  software  vendor  provided  the  control  and  management 
(C&M)  software  (PMW173  2002,  18-19). 
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Figure  6.  CSRR  Operational  View  (OV-1)  (from  PMW770  2006,  A.  1.1) 


PMW173  was  assigned  the  responsibilities  as  the  lead  systems  integrator  for  the  CSRR 
program.  Following  the  initial  delivery  for  the  VA,  the  approach  has  been  replicated  on 
the  SSGN,  SSBN  and  SW  classes.  Currently,  CSRR  is  deployed  in  several  versions 
across  all  classes  (e.g.,  SSBNs  have  increment  one  version  one  (VI),  SSGN  and  SW 
classes  have  increment  one  version  two  (V2),  and  VA  has  increment  one  versions  one, 
two  and  three  (V3).  In  2012  the  LA  class  installed  the  first  CSRR  on  the  USS  Hampton. 
The  approach  has  been  extended  to  the  Ohio  SSBN  replacement  program  (ORP). 

CSRR  leveraged  the  benefits  of  bundling  the  capabilities  of  other  established 

acquisition  programs  of  record  (POR)  and  integrating  them  using  an  open  systems 

architecture  system  of  systems  core  design  approach.  Section  1.3.3  of  the  CPD  (PMW770 

2006,  3)  provides  a  general  description  of  the  functional  end-to-end  communications 

integration.  CSRR  integrates  the  program  of  record  component  systems,  makes  any 

necessary  modifications  to  accommodate  any  related  support  equipment  and  perform 

coordinated  development,  testing,  and  installation  of  the  new  capabilities.  The  design  and 

development  is  accomplished  in  a  serial  fashion,  with  a  version  completed  for  each  class 

before  beginning  development  of  the  next.  The  installation  of  CSRR  on  the  LA  platforms 
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identified  the  need  to  evolve  this  approaeh  in  order  to  preserve  operational  availability 
and  add  fiexibility  with  other  PORs. 

The  baseline  system  met  the  submarine  exterior  eommunieations  requirements 
defined  in  the  CSRR  CPD  and  SUBECS  CRD.  The  CSRR  Systems  Engineering  Plan 
(PMW770  2007)  points  out  CSRR  program  key  performanee  parameters  are  dependent 
on  the  eapabilities  of  the  system  that  aetually  makeup  a  version.  Capability  upgrades 
were  planned  to  oeeur  as  ehanges  oeeurred  in  other  programs  of  record  (POR)  including 
automated  digital  networking  system  increment  three  (ADNS  Inc3),  Navy  multi-band 
terminal  (NMT),  joint  tactical  radio  system  (JTRS),  and  mobile  user  objective  system 
(MUOS).  Each  program  maintains  its  own  acquisition  responsibilities.  CSRR  integrates 
these  systems  into  the  overarching  architecture,  provide  updated  C&M  software,  and 
creates  system  level  documentation  and  training.  These  capabilities  would  fit  within  the 
CSRR  open  system  architecture.  A  sample  of  the  CSRR  architecture  is  shown  in  Eigure 
7. 


2,  CSRR  Program  Description 

The  CSRR  program  is  managed  by  the  Undersea  Integration  Program  Office 
(PMW770)  within  PEO  C4I.  PMW770  is  the  designated  lead  integrator  for  systems 
destined  to  be  installed  onboard  a  submarine  or  submarine  broadcast  control  authority 
(BCA).  Since  a  submarine  C4I  SOS  initial  capabilities  document  does  not  exist  for 
submarine  communications  the  CSRR  program  must  work  closely  with  the  other 
programs  in  order  to  achieve  the  operational  requirements  identified  in  the  CPD 
(PMW770  2006,  6). 

As  an  ACAT  II  program  CSRR  has  the  responsibilities  as  the  lead  systems 
integrator.  In  order  to  perform  this  responsibility  CSRR  is  closely  engaged  with  several 
ACAT  I  major  defense  acquisition  programs  such  as  the  NMT,  GBS,  MUOS,  VA  and  the 
Ohio  replacement  program.  The  challenge  is  working  with  the  large  and  small  programs 
to  carry  out  the  duties  as  the  lead  systems  integrator. 
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Figure  7.  Common  Submarine  Radio  Room  (from  Anderson  2014) 


In  this  role,  CSRR  not  only  integrates  systems  but  coordinates  the  activities  of 
other  organizations  and  teams.  Figure  8  is  the  organization  structure  of  the  Undersea 
Integration  Program  Office.  The  highlighted  area  is  the  personnel  assigned  specifically  to 
the  CSRR  program.  Personnel  assigned  to  the  other  divisions  work  closely  with  the 
CSRR  program  and  other  programs  of  record  (POR)  to  manage  requirements,  integration, 
testing,  fielding,  and  sustainment  responsibilities. 
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Figure  8.  PMW770  Program  Office  Structure  (after  Anderson  2014) 


The  CSRR  program  team  includes  a  number  of  external  stakeholders  responsible 
for  the  various  functional  areas  shown  in  Table  3.  These  include  the  OPNAV  resource 
sponsors  within  the  CNO’s  office,  the  ships’  program  managers  within  Naval  Sea 
Systems  Command  (NAVSEA),  the  user  community  represented  by  COMSUBFOR, 
Space  and  Naval  Warfare  Systems  Command  (SPAWAR),  SPAWAR  Fleet  Readiness 
Directorate  (FRD),  Space  and  Naval  Warfare  Systems  Center  Atlantic  (SSC  FANT), 
Space  and  Naval  Warfare  Systems  Center  Pacific  (SSC  PAC),  Naval  Undersea  Warfare 
Center  (NUWC)  Newport,  and  Submarine  Feaming  Center.  The  specific  relationships  are 
shown  in  Table  3.  Figure  9  shows  the  relationships  between  the  internal  and  external 
stakeholders  supporting  the  CSRR  program.  The  organizations  inside  the  circle  are 
closely  teamed  with  the  CSRR  program.  This  relationship  extends  into  the  design  and 
production  groups  as  well  the  sustainment  and  training  activities  in  order  to  develop 
synergy.  Installations  are  accomplished  in  a  coordinated  approach  managed  by  the  FRD 
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through  their  installation  management  offices  located  within  SSC  LANT  and  SSC  PAC. 
The  SPAWAR/PEO  modernization  CONOPS  (PEO  C4I  2005)  details  how  the  individual 
programs  and  the  platform  program  offices  will  coordinate  their  efforts  to  accomplish 
design,  development  and  modernization. 

Tables.  CSRR  Stakeholders 


Stakeholder 

Relationship 

CNO  OPNAVN2/N6 

Resource  sponsor — Provides  funding  and 
requirements 

NAVSEASYSCOM 

Ships  Acquisition  Platform  Manager 

COMSPAWARSYSCOM 

Eunctional  and  matrix  support  for  logistics, 
systems  engineering,  and  acquisition 

SPAWAR  Elect  Readiness  Directorate 

Installation  management  and  sustainment  of 
systems  past  full  rate  production 

SPAWAR  Systems  Center  Atlantic 

Production  management;  sustainment  of 
in-service  systems;  training  development 

SPAWAR  Systems  Center  Pacific 

Control  and  management  software 
development  and  management 

Naval  Undersea  Warfare  Center 

Division  Newport 

Design  and  testing;  documentation 
development 

Submarine  Learning  Center 

Training  delivery,  formal  classroom  and 
modernization  training 

Commander  Submarine  Eorces 

End  user,  requirements  generator 
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Figure  9.  CSRR  Program  Model  (from  Anderson  2014) 


Revision  one  of  the  Acquisition  Plan  /  Acquisition  Strategy  (AP/AS)  (PMW770 
2008,  13)  describes  CSRR  as  a  system  of  systems  which  integrates  Navy  PORs.  CSRR 
integrates  the  systems  from  other  programs  of  record  such  as  EHF,  GBS,  ADNS, 
submarine  masts  and  antennas  outboard  electronics  (OF)  OE-538/OE-592,  OE-562, 
periscopes,  floating  wire  antennas,  towed  buoy  antennas,  submarine  single  messaging 
system  (SUBSMS),  digital  modular  radio  (DMR)  and  others.  Each  system  provides  an 
aggregate  component,  which  when  integrated  creates  a  system  of  systems.  Each 
individual  system  has  its  own  program  schedules  and  required  capabilities  but  when 
integrated  together  achieve  capabilities  not  possible  as  an  individual  component. 

The  CSRR  test  and  evaluation  master  plan  (TEMP)  (PMW770  2012)  discusses  all 
of  the  testing  accomplished  previous  to  V3  and  identifies  the  overarching  plan  for  testing 
of  capabilities  delivered  with  V3.  Section  1.3.3  related  to  key  capabilities  and  interfaces, 
discusses  the  necessity  of  CSRR  as  a  system  of  systems  requirement  to  interface  the 
component  systems,  host  platform  systems,  other  DOD  components,  and  allied  and 
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coalition  partners.  The  speeific  performanee  requirements  and  eharacteristies  of  eaeh 
component  system  are  defined  within  their  respective  program  documents. 

The  CSRR  program  requirements  are  defined  in  the  CSRR  Capabilities 
Production  Document  (CPD)  (PMW770  2006)  and  SUBECS  CRD  (CNO  N8  2003).  The 
CPD  was  developed  in  support  of  the  production  decision  for  increment  one  Version  zero 
(VO).  The  CPD  in  coneert  with  the  CRD,  AP/AS,  system  engineering  plan  and  TEMP 
outline  the  main  requirements  for  developing  and  deploying  each  successive  version  of 
CSRR. 

Changing  teehnology  presents  challenges  to  system  of  systems  programs  since 
most  aequisition  doeumentation  is  ereated  at  the  beginning  of  the  program  and  placed  on 
the  shelf  after  aehieving  the  produetion  and  deployment  phase.  The  CSRR  circuit  matrix 
(PMW770  2014)  is  an  agreement  maintained  between  PMW770  and  the  submarine  force 
to  capture  changes  delivered  by  new  programs  of  record  and  changes  to  operational 
doetrine.  The  eircuit  matrix  is  a  living  doeument  that  is  periodically  reviewed  and 
updated  to  reflect  the  evolving  communieations  eapabilities  for  eaeh  CSRR  version. 

CSRR  also  identified  the  need  to  maintain  a  eommon  “core  eapability”  (PMW770 
2006,  34),  which  acknowledges  there  are  differences  in  the  submarine  platforms  but  the 
overall  mission  requirements  remain  the  same.  Maintaining  the  architecture,  workstation 
and  interfaces  common  across  all  classes  provide  the  basis  for  the  eore  capabilities.  This 
same  core  eapability  is  used  in  support  of  the  incremental  development  approaeh.  Onee 
the  eore  capabilities  have  been  identified  they  form  the  baseline.  This  baseline  allows  for 
scalability  and  modularity.  This  core  capability  approach  was  briefed  to  the  milestone 
decision  authority  during  a  gate  six  review  which  resulted  in  maintaining  CSRR  at 
inerement  one  with  eaeh  version  providing  new  capabilities  from  other  programs. 

The  CSRR  Requirements  Design  Integration  Test  Process  (Ross  2013)  provides 
an  introduction  for  new  personnel  to  get  aequainted  with  the  CSRR  program  and  for 
experienced  personnel  to  have  a  ready  reference.  The  doeument  was  generated  as  a 
Capability  Maturity  Model  Integration  (CMMI)  initiative  to  eapture  the  proeesses  used 
within  the  CSRR  program  management,  engineering  and  test  teams. 
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c. 


CSRR  CONCEPT  OF  OPERATIONS 


A  concept  of  operations  (CONOPS)  is  a  vision,  verbally  or  using  graphics,  of  how 
a  system  is  expected  to  be  employed  by  the  warfighter.  The  JCIDS  (JCS  2012)  and  Joint 
Pub  (JP)  5-0  (JCS  2011)  outlined  the  purpose  of  a  CONOPS  is  to  illustrate  how  a  joint 
foree  commander  will  organize  his  forces  and  deploy  them  for  a  particular  scenario  or  in 
support  of  the  introduction  of  new  capabilities.  From  the  CONOPS  the  acquisition 
community  can  decompose  a  mission  concept  into  its  constituent  components  and  begin 
defining  how  to  test  and  deploy  once  it  is  ready.  The  CSRR  CONOPS  describes  the 
different  systems  and  capabilities  available  for  each  version.  These  capabilities  are  shown 
in  the  various  scenarios  the  submarine  would  be  reasonably  expected  to  exeeute.  There 
are  eight  scenarios  developed  for  the  CONOPS.  Each  of  these  scenarios  describes  how 
CSRR  will  be  employed  from  initial  deployment  through  the  post  event  reporting 
aetivities.  Figure  10  is  a  simplified  graphic  of  the  systems  composing  CSRR  and  its 
relationship  to  the  external  systems. 


Figure  10.  CSRR  High  Fevel  Concept  Graphic  (from  PMW770  2011b,  2) 
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Since  CSRR  is  an  SOS  the  CONOPs  show  the  aggregate  C4I  capabilities  needed 
to  support  the  following  mission  scenarios: 

1 .  Land  attack/strike  mission  (STK) 

2.  Intelligence,  surveillance  and  reconnaissance  mission  (ISR) 

3.  Carrier  strike  group/expeditionary  strike  group  operations  mission 
(CSG/ESG) 

4.  Special  operations  forces  mission  (SOF) 

5.  Mine  warfare  operations  mission  (MIW) 

6.  Undersea  warfare  mission  (USW) 

7.  Surfaee  warfare  mission  (SUW) 

8.  Strategic  deterrence  mission  (SD) 

Eaeh  mission  scenario  proceeds  through  the  pre-deployment  to  post  mission 
reporting.  Most  scenarios  share  eommon  pre  and  post  mission  activity  characteristics  but 
interfaces  with  different  activities  and  may  use  different  primary  and  secondary 
communications  paths. 

1.  Land  Attack/Strike  Mission 

The  STK  scenario  describes  the  aetivities  that  oecur  in  support  of  launehing 
Tomahawk  missiles.  The  STK  CONOPS  shown  in  Figure  11  describes  the  CSRR 
activities  that  oecur  during  each  phase  by  providing  the  voice,  video  and  data  pathways 
necessary  for  coordinated  land  attaek/strike  operations.  Eaeh  line  identifies  the  type  of 
communieations  available  in  eaeh  phase.  Data  flows  are  a  key  element  for  STK  refleeted 
in  a  majority  of  the  phases. 

2,  Intelligence,  Surveillance  and  Reconnaissance  Mission 

Submarines’  stealth  makes  them  ideally  suited  for  ISR  missions.  CSRR  enables 
communieations  with  in-theater,  national  command,  or  intelligence  eommunity  activities 
to  coordinate  the  entire  spectrum  of  ISR  operations  during  peaeetime  or  hostilities 
including  coordination  of  STK  or  SOF  missions  in  hostile  areas.  More  emphasis  is  on 
eovertness  and  exchange  of  intelligence  information,  including  imagery. 
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Figure  1 1 .  Land  Attack/Strike  Mission  Scenario  (from  PMW770  2011b,  52) 


3,  Carrier  Strike  Group/Expeditionary  Strike  Group  Operations 
Mission 


Attack  submarines  support  CSG/ESG  operations.  CSRR  provides  voice,  video 
and  data  paths  for  coordinated  operations  with  joint  task  forces,  group  and  other 
combatant  commanders.  This  requires  the  submarine  to  communicate  in  a  stealthy  mode 
to  maximize  its  search  capabilities.  More  voice  circuits  are  needed  in  concert  with  the 
data  flow.  Figure  12  shows  how  a  submarine  coordinates  with  the  CSG/ESG  to  provide 
support  adversaries. 
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Simullaneity  Point 


Figure  12.  CSG/ESG  Mission  Scenario  (from  PMW770  2011b,  67) 


4,  Special  Operations  Forces  Mission 

Submarines  are  effective  platforms  for  supporting  SOF  operations  for  mission 
planning,  insertion,  coordination  and  extraction.  Communications  emphasizes  tactical 
data  and  voice  circuits  required  to  coordinate  with  embarked  SOF  commanders,  naval 
computer  and  telecommunications  area  master  station,  submarine  BCA  and  joint  special 
operations  task  force. 

5,  Mine  Warfare  Operations  Mission 

Submarines  are  capable  of  deploying  mines  to  deny  sea  areas  as  well  as  mapping 

detected  minefields  and  communicate  the  information  to  other  units.  CSRR  supports 

31 


execution  of  this  capability  through  reporting  detected  minefields.  The  MIW  mission  is 
similar  to  the  CSG/ESG  operations  mission.  The  operational  nodes  and  communications 
paths  are  the  same  but  data  versus  voice  is  sent  over  the  communication  lines. 

6,  Undersea  Warfare  Mission 

USW  against  hostile  submarines  is  the  traditional  submarine  mission.  Typically 
operating  independently,  coordination  with  surface  and  air  units  creates  a  need  for 
common  communications  during  the  detection  and  tracking  of  enemy  submarines.  CSRR 
provides  the  capability  to  receive  intelligence  of  detected  submarines  when  in  a  covert 
mode  or  pass  intelligence  to  units  assuming  the  track  or  prosecution  of  hostile  units. 
CSRR  also  provides  communications  links  to  support  pre -mission  operational 
preparation,  common  operational  picture  (COP)  updates,  situation  reports  to  the  USW 
commander  and  tasking  by  the  USW  Commander.  The  operational  nodes  used  for  the 
USW  mission  are  the  same  as  the  CSG/ESG  and  MIW  missions.  The  USW  mission 
emphasis  is  on  data  paths  that  support  the  submarine  being  deep  for  extended  periods  of 
time,  maximizing  search  effectiveness.  USW  aircraft  also  play  a  significant  role  in  this 
mission,  and  therefore  they  were  included  as  part  of  the  CSG/ESG  commander  node. 

7.  Surface  Warfare  Mission 

SUW  is  a  collateral  independent  mission  to  track  and  destroy  lone  surface  units 
without  assistance.  However,  this  opportunity  is  not  available  when  groups  of  surface 
combatants  are  in  the  same  area.  The  danger  of  counter-detection  with  limited  evasion 
possibilities  makes  this  scenario  a  cautious  one  for  submarines.  If  it  is  not  possible  for  the 
submarine  to  conduct  a  direct  attack,  then  it  can  assist  or  coordinate  attacks  on  surface 
units.  CSRR  provides  communications  links  to  support  pre-mission  operational 
preparation,  COP  updates,  situation  reports  to  and  tasking  from  the  SUW  commander. 
The  operational  nodes  and  data  flows  are  the  same  as  the  USW  mission.  The  SUW 
mission  requires  greater  coordination  with  other  units  to  prevent  engaging  friendly 
contacts  and  to  feed  information  into  the  COP. 
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8,  Strategic  Deterrence  Mission 

SSBNs  makeup  one  leg  of  the  nuclear  triad  remaining  submerged  to  avoid 
detection  while  on  alert.  Reliable  communications  is  a  key  requirement.  Should  a  missile 
launch  be  ordered  the  SSBN  will  receive  their  orders  via  an  emergency  action  message 
(EAM).  Common  Submarine  Radio  Room  receives  intelligence,  situation  reports,  and 
EAMs  while  operating  in  a  covert  manner.  Additionally,  CSRR  supports  communications 
links  for  pre-mission  operational  preparation,  COP  updates,  targeting  change  messages 
(TCM),  situation  reports  and  tasking.  Eigure  13  depicts  the  nuclear  command,  control  and 
communications  (NC3)  infrastructure  needed  for  mission  communications  while 
performing  a  strategic  deterrent  patrol.  The  BCA  provides  the  interface  to  the  NC3 
system  for  delivery  of  EAMs.  Take  charge  and  move  out  (TACAMO)  aircraft  and  surface 
ships  relay  EAMs  if  there  is  a  failure  of  the  primary  reception  paths.  The  simultaneity 
point  indicates  when  multiple  communications  paths  will  be  available  for  use. 


Eigure  13.  Strategic  Deterrence  Mission  Scenario  (from  PMW770  2011b,  106) 
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D. 


SYSTEMS  COMPRISING  CSRR 


Section  2.4  of  the  revised  CSRR  Acquisition  Strategy  /  Acquisition  Plan  (AS/AP) 
(PMW770  2008)  defines  CSRR  as  a  network-centric  communications  system  of  systems, 
integrating  several  program  of  record  systems  within  a  common  architecture  to  provide 
secure,  reliable,  and  covert  communications  and  effectively  manage,  control,  process  and 
disseminate  command,  control,  computers,  intelligence,  surveillance  and  reconnaissance 
(C4ISR)  information.  Systems  engineering  activities  during  the  development  phase 
address  issues  to  align  the  individual  system  requirements  with  the  CSRR  SOS 
requirements.  The  program  office  continually  engages  with  the  PORs  to  integrate  and 
deploy  new  and  updated  capabilities  in  an  evolutionary  manner.  CSRR  is  composed  of 
systems  from  the  following  program  offices  as  shown  in  Figure  14. 
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Figure  14.  CSRR  Program  Relationships  to  Other  Programs  of  Record 
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The  CSRR  program  shares  product  development  and  integration  responsibilities. 
Within  this  scope  CSRR  integrates  the  individual  component  systems  procured  via  their 
parent  program  offices,  shown  in  Figure  15,  and  procures  or  modifies  subsystem 
components  and  ancillary  equipment  (e.g.,  racks,  rack  cabling,  and  routers)  into  a 
common,  open  architecture  baseline,  with  control  and  management  of  the  physical 
components  provided  by  the  CSRR  C&M  software.  The  AS/AP  directs  how  CSRR 
program  must  approach  the  modernization  of  each  version  for  considering  technology 
insertion  activities  to  integrate  new  products  and  capabilities. 

The  open  system  architecture  maximizes  the  use  of  COTS,  allows  for  the 
rapid  insertion  of  technology,  and  addresses  emerging  requirements  and/or 
obsolescence  issues.  This  design  flexibility  is  particularly  important  in  this 
submarine  communication  program  due  to  changing  requirements  and 
emerging  technical  advances. 

Use  of  non-proprietary  standards  and  protocols  enhance  the  program's 
ability  to  efficiently  and  effectively  respond  to  requirements  changes, 
incorporate  commercial  system  improvements,  and  improve 
interoperability  within  the  GIG. 

The  CSRR  modernization  plan  will  continue  to  apply  open  system 
architecture  concepts  and  plans  for  platforms  with  technology  refresh  back 
fits  concurrent  with  modernization  increments.  (PMW770  2008,  26) 

Baseline  updates  are  closely  managed  and  accomplished  as  minor  upgrades  or  in 
concert  with  a  larger  version  modernization  effort.  Updates  include  C&M  software 
changes  incorporating  new  functionality  or  capability.  The  Gate  Six  review  (PMW770 
2008,  1-2)  authorized  a  change  to  the  AS/AP  defining  version  vice  increment  upgrades. 
The  significance  of  this  decision  eliminated  the  requirements  to  create  new  acquisition 
documentation  but  did  direct  each  version  to  accomplish  operational  testing  (OT).  The 
extent  of  testing  is  negotiated  with  Director,  Operational  Test  and  Evaluation  (DOT&E), 
and  Commander,  Operational  Test  and  Evaluation  Eorce. 
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Figure  15.  Programs  of  Record  Systems  Composing  CSRR 


Testing  accomplished  by  individual  programs  of  record  will  be  leveraged  where  possible, 
however  the  main  intent  is  determine  if  CSRR  as  a  system  of  systems  still  meets 
requirements  and  is  deemed  operationally  effective  and  suitable.  The  following  program 
offices  provide  their  products  for  integration  and  testing  into  a  CSRR  version. 

1,  PMW  130  Information  Assurance  and  Cybersecurity  Program  Office 

PMW  130  provides  cyber  security  products  and  services  and  cryptographic 
products  to  protect  Navy  and  Marine  Corps  C41  systems  (PEO  C41  2012).  PMW  130 
provides  the  following  systems. 
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a.  Crypto  Universal  Enclosure 

The  crypto  universal  enclosure  (CUE)  provides  a  common  host  for  the  various 
modem  crypto  devices. 

b.  Electronic  Key  Management  System 

The  electronic  key  management  system  (EKMS)  handles  the  administrative  and 
key  generation  capabilities  onboard  the  platform. 

c.  Cryptographic  Devices 

There  are  a  number  of  cryptographic  devices  required  to  support  secure 
communications  across  the  various  communications  and  IP  networks.  Most  of  these 
devices  are  hosted  in  the  CUE. 

2,  PMW  160  Tactical  Networks  Program  Office 

PMW  160  manages  the  network  programs  for  afloat,  airborne,  and  ashore  nodes 
(PEO  C4I  2012).  These  include  the  Automated  Digital  Networking  System  (ADNS), 
Sensitive  Compartmented  Information  (SCI)  and  Special  Intelligence  (SI)  Network 
System,  and  Submarine  Eocal  Area  Network  (SUBLAN). 

a.  Automated  Digital  Network  System 

The  Automated  Digital  Network  System  (ADNS)  is  an  ACAT  III  program 
managed  within  the  Tactical  Networks  Program  Office  and  provides  the  main  access 
point  to  the  Navy  tactical  /  strategic  and  global  information  grid  resources  and  services. 
ADNS  provides  wide  area  network  (WAN)  connectivity  and  is  the  Navy’s  bandwidth 
optimization  program  of  record  to  provide  quality  of  service  routing  for  voice,  video  and 
data  using  the  available  communications  links  within  the  ship  /  shore  WAN.  ADNS 
interfaces  to  the  various  Navy  networks  to  enable  interfaces  to  U.S.  classified  and 
unclassified  networks,  and  Allied  and  Coalition  networks  (General  Dynamics  2008). 
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b.  Sensitive  Compartmented  Information  Networks 

The  primary  mission  of  sensitive  compartmented  information  (SCI)  networks 
provides  connectivity  to  the  intelligence  community  to  provide  shipboard  analysts  with 
access  to  national  and  service  strategic  and  tactical  databases.  SCI  networks  is  the 
transport  medium  providing  special  intelligence  data  and  secure  WAN  IP  access  to  ship 
and  shore  national  Web  sites,  signals  intelligence  and  intelligence  databases  for  seamless 
interaction  between  shore,  surface,  submarine  and  airborne  special  intelligence  LANs 
(PEG  C4I  2012). 


c.  Submarine  Local  Area  Network 

Submarine  Local  Area  Network  (SUBLAN)  is  a  reliable  high-speed  secret, 
sensitive  but  unclassified  and  top  secret  local  area  network  (PEG  C4I  2012).  When  the 
SUBLAN  network  is  combined  with  other  subsystems,  it  provides  the  shipboard  network 
services  and  uses  CSRR  as  the  gateway  for  off  hull  services  to  deliver  an  end-to-end  net 
centric  warfare  capability.  The  Consolidated  Afloat  Network  Enterprise  System 
(CANES)  is  the  next  generation  network  planned  for  afloat  units  to  consolidate  many  of 
the  individual  networks  into  a  larger  system  of  systems  network  (PEG  C4I  2014). 

3,  PMW/A  170  Communications  and  Navigation  Program  Office 

PMW/A  170  provides  satellite,  line-of-sight,  and  extended-line-of-site 
communication  systems  for  voice  and  data  communications  and  Global  Positioning 
System  (GPS)  capabilities  for  ship  navigation,  command  and  control  systems  and 
weapons  systems  (PEG  C4I  2012).  PMW  170  oversees  the  Navy  EHE  satellite  program, 
DMR,  Global  Broadcast  Service  (GBS),  Time  Division  Multiple  Access  Interface 
Processor  (TIP),  and  Portable  Radio  Program. 

a.  Military  Strategic  and  Tactical  Relay  System  /  Navy  Extremely 
High  Frequency  Program  /  Navy  Multiband  Terminal 

The  military  strategic  and  tactical  relay  system  (MILSTAR)  EHE  system  was 
introduced  in  the  1980s  as  an  answer  to  the  challenges  facing  users  of  the  UHE  band  to 
provide  tactical  and  strategic  communications  in  all  environments.  Bandwidth  capability 
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has  grown  since  its  introduction  from  2400  bps  to  over  24  Mbps.  The  Navy  Multiband 
Terminal  (NMT)  is  the  latest  maritime  military  satellite  eommunications  terminal 
supporting  the  Military  Satellite  Communication  (MILSATCOM)  architecture  to  provide 
conneetivity  in  all  domains.  NMT  supports  the  advaneed  extremely  high  frequeney 
program  for  proteeted  satellite  eommunications.  The  NMT  eommunieates  via  the 
Wideband  Global  Satellites  and  Defense  Satellite  Communications  System  (DSCS)  for 
super  high  frequency  (SHF).  NMT  operates  over  EHF  low  data  rate,  medium  data  rate, 
and  extended  data  rate  communieation  modes  (PEO  C4I  2012). 

b.  Global  Broadcast  Service 

Global  Broadcast  Service  is  a  joint  program  led  by  the  Air  Eoree  to  provide  high 
bandwidth  capability  to  deliver  classified  and  unclassified  products.  Classified  as  an 
ACAT  1  program,  GBS  leverages  the  EHE  system  as  a  transport  for  a  “smart  push”  and 
“user  pull”  approaeh  for  delivering  products.  Data  can  include  full  motion  video, 
imagery,  maps,  orders,  and  weather  information.  Unified  eommanders  manage  the  flow 
of  these  products  over  the  portions  of  the  system  supporting  their  area  of  responsibility. 
GBS  uses  CSRR  and  ADNS  to  route  information  to  the  elassified  and  unclassified 
computers.  Eive  video  feeds  are  sent  to  the  submarines  training  and  entertainment  system 
for  viewing  by  the  crew  (CNO  N61  and  N87  1998,  3-13). 

c.  Digital  Modular  Radio 

Digital  Modular  Radio  is  a  multi-channel  software  programmable  radio  eapable  of 
operating  aeross  the  HE-UHE  frequency  spectrum  and  is  interoperable  and  eompatible 
with  legaey  systems  (PEO  C4I  2012;  General  Dynamics  2014).  The  DMR  is  the  COTS 
replacement  for  the  MINI-DAMA. 

d.  Miniature  Demand  Assign  Multiple  Access 

The  MINI-DAMA  is  a  legaey  two-ehannel  VHE/UHE  radio  retained  in  the  LA 
class  CSRR  Increment  I  V3.  Built  in  the  1990s,  the  MINI-DAMA  provides  UHE 
SATCOM  and  LOS  eapability  and  ineorporates  the  Advaneed  Digital  Waveform  (ADW) 
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to  support  the  Medium  Data  Rate  Channel  Aeeess  Protoeol  (MCAP)  eireuit  (Federation 
of  Ameriean  Seientists  [FAS]  1999). 

e.  MD-1324  Advanced  Digital  Waveform  Modem 

Early  versions  of  DMR  do  not  have  the  ADW  waveform  integrated  as  an  organie 
eapability.  In  order  to  support  improved  throughout  the  MIL  standard  for  UHF 
waveforms  was  updated  to  inelude  ADW.  The  MD-1324  has  the  ADW  waveform  to 
support  MCAP  by  interfacing  the  modem  with  the  DMR  power  amplifiers. 

/.  PSC-5D  Integrated  Waveform  Radio 

The  integrated  waveform  (IW)  was  developed  by  the  Defense  Information 
Systems  Agency  as  a  solution  to  diminishing  UHF  resources.  The  PSC-5D  is  a 
commercial  radio  installed  on  LA  platforms  to  provide  IW  capability. 

4,  PMW770  Undersea  Integration  Program  Office 

PMW770  is  responsible  for  the  development,  acquisition,  and  integration  and 
fielding  of  systems  planned  for  the  undersea  domain  (PEO  C4I  2012).  They  manage 
product  and  integration  responsibilities  for  the  following  programs:  Radio  Lrequency 
Distribution  and  Control  System  (RLDACS),  Q-70  and  Novo  workstations,  control  and 
management  (C&M)  software,  SLVR,  OE-538  Multifunction  Mast,  OE-562  Submarine 
High  Data  Rate  (SUBHDR),  BRR-6  towed  buoy  antenna  and  OE-315  floating  wire 
antenna. 


a.  Radio  Frequency  Distribution  and  Control  System 

The  RLDACS  provides  an  automated  interface  between  the  radios  and  submarine 
antenna  systems,  amplifying  and  distributing  RE  frequencies  to  various  systems  such  as 
GPS. 
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b.  Q-70  and  Novo  CSRR  Workstations 


The  workstations  provide  the  human-machine  interface  between  the  operator  and 
CSRR.  The  C&M  software  provides  the  graphical  user  interface  to  align  and  operate  the 
various  communications  circuits  available  to  the  operator. 

c.  RT-9000  HF  Transceiver 

The  RT-9000  is  a  COTS  radio  installed  on  LA  class  submarines.  The  radio 
provides  HF  voice  and  data  capability. 

d.  CSRR  Ancillary  Equipment 

Ancillary  equipment  consists  of  HF  modems,  secure  voice  switch,  black  audio 

switch. 


e.  Submarine  LF/VLF  Versa  Module  Eurobus  Receiver 

SLVR  is  the  VLF/LF  receiver  capable  of  receiving  and  processing  all  Navy, 
special,  and  NATO  modes.  SLVR  receives  message  traffic  from  the  FSBS  via  one  of 
several  VLF  antennas  while  submerged.  The  SLVR  is  installed  on  all  submarines  and  is 
operated  from  the  CSRR  operator  work  station.  SSBNs  have  additional  capability  to 
directly  control  the  SLVR  via  a  local  processor  and  can  send  messages  directly  to  a 
printer  for  handling  (CNO  N61  and  N87  1998). 

/.  Time  and  Frequency  Distribution  System 

The  TFDS  provides  precision  time  and  frequency  information  to  communications, 
electronic  warfare,  periscope,  navigation,  combat,  and  ship  control  systems  aboard  attack 
and  Trident  class  submarines.  The  TFDS  is  a  NDI  system  using  two  rubidium  standards 
which  eliminate  single  points  of  failure.  The  TFDS  can  select  inputs  from  the  internal 
standards  or  an  external  device  such  as  GPS.  The  TFDS  provides  a  variety  of  outputs  for 
frequency  and  timing  and  time  code.  The  TFDS  originated  as  a  Submarine  Integration 
Program  Office  as  an  AC  AT  IVT  program.  In  2011  the  FRD  assumed  sustainment 
responsibilities. 
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g.  OE-538  Multi-Function  Mast  Antenna  Group 

The  OE-538  antenna  group  is  an  improved,  multifunctional,  combined 
communications,  navigation  and  Identification  Friend  or  Foe  (IFF)  mast-mounted 
antenna  system  for  all  submarine  classes.  The  OE-538  covers  the  RF  spectrum  for  all 
VFF  to  UHF  requirements  including  IFF  and  GPS  and  provides  significant  reliability 
improvements  (CNO  N61  and  N87  1998).  Increment  2  expands  the  capability  to  include 
support  for  the  Mobile  Users  Objective  System  (MUOS),  iridium,  and  Tactical  Data  Fink 
via  Fink  16. 


h.  Submarine  High  Data  Rate  Antenna  System 

The  SUBHDR  antenna  provides  connectivity  for  the  EHF,  SHF  and  GBS 
communications  for  submarines  capable  of  supporting  data  rates  up  to  eight  megabits  per 
second.  The  SUBHDR  antenna  uses  a  16-inch  dish  antenna  which  is  controlled  by  the 
EHF  terminal  to  point  the  satellite. 

5.  PMW  790  Shore  and  Expeditionary  Systems  Program  Office 

PMW  790  manages  the  Navy’s  messaging  systems  to  provide  the  message 
handling  and  distribution  responsibilities  for  afloat  and  shore  activities  (PEG  C4I  2012). 
Within  PMW  790  is  the  submarine  single  messaging  system  (SUBSMS)  which  is 
composed  of  the  following: 

a.  Navy  Modular  Automated  Communications  System  II 

The  Navy  modular  automated  communications  system  model  (NAVMACS)  II  is 
a  ship  to  ship,  ship  to  shore  messaging  system  which  handles  organizational  message 
traffic  and  routes  it  to  the  ships  FAN  for  distribution  to  the  appropriate  mailbox.  In  event 
a  high  priority  message  is  received  it  can  be  configured  to  print  out  a  copy  for  immediate 
action.  NAVMACS  is  set  up  primarily  to  handle  incoming  serial  traffic  transmitted  by  the 
FSBS  or  MIFSTAR  (NUWC  2012,  194-196). 
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b.  Submarine  Single  Messaging  System  Support  Server 

The  SUBSMS  support  server  (SSS)  is  the  main  system  to  send  and  receive 
organizational  message  traffic  received  over  an  IP  broadcast.  The  SUBSMS  interfaces 
with  the  NAVMACS  II  to  manage  message  reception,  processing,  storage  and 
transmission  (NUWC  2012,  I94-I96).  Additionally,  SUBSMS  hosts  the  Information 
Screening  and  Delivery  System  which  handles  messages  received  via  IP  paths  and  routes 
them  to  the  various  users  on  the  SUB  LAN. 

6,  Activities  External  to  PEO  C4I 

CSRR  interfaces  with  multiple  systems  managed  by  other  program  offices  within 
PEO  SUB  and  Program  Executive  Office  Integrated  Warfare  Systems  (PEO  IWS)  under 
NAVSEA,  Nuclear  Command  and  Control  under  Strategic  Systems  Programs  (SSP), 
strike  and  airborne  relay  via  Naval  Air  Systems  Command  (NAVAIR),  and  submarine 
force  sponsored  installations  and  alterations. 

a.  Program  Executive  Office  Submarines 

PEO  SUB  manages  the  submarine  warfare  federated  tactical  system  (SWETS). 
SWETS,  shown  in  Eigure  16,  is  an  SOS  model  composed  of  the  combat  control,  sensors, 
SUBLAN,  shown  in  yellow,  and  CSRR  managed  under  a  mutual  agreement  between  the 
systems  commands.  CSRR,  shown  in  pink,  is  considered  a  system  within  this  larger  SOS. 
These  systems  are  managed  within  NAVSEA  under  PEO  SUB  and  PEO  IWS  and 
program  management  air  (PM A)  271  under  NAVAIR.  NAVSEA  has  responsibility  as  the 
ship  acquisition  program  manager  for  oversight  of  all  submarines  acquired  and  currently 
in  service. 


b.  Strategic  Systems  Programs 

Strategic  Systems  Programs  has  oversight  of  all  NC3  systems.  CSRR  interfaces 
with  the  NC3  community  through  SSP  and  their  agent  Naval  Surface  Warfare  Center 
Dahlgren  to  certify  messaging  paths  responsible  for  delivery  of  nuclear  command  and 
control  information. 
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Figure  16.  Submarine  Warfare  Federated  Taetieal  System 

(after  PMW770  2012b) 


E.  SYSTEMS  ENGINEERING  AND  SYSTEM  OF  SYSTEMS  ENGINEERING 
PRINCIPLES 

DOD  eontinuously  develops  and  procures  systems  to  support  the  warfighter. 
These  systems  must  meet  their  requirements  in  terms  of  cost,  schedule,  and  performance. 
Sound  systems  engineering  (SE)  is  critical  to  design,  deliver,  and  support  complex 
system  of  systems  in  order  to  achieve  these  key  tenets.  The  number  of  attempts  to 
institute  acquisition  reform  during  the  last  30  years  has  proven  challenging  with  few 
substantial  changes  really  occurring.  The  Weapons  Systems  Acquisition  Reform  Act  of 
2009  provided  several  changes  to  include  mandating  use  of  SE  and  establishing  the 
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Deputy  Assistant  Secretary  of  Defense  for  Systems  Engineering  (DASD  SE).  The 
Government  Accounting  Office  annual  high  risk  report  (GAO  2013,  151)  reported  the 
following 

Moving  forward,  DOD  faces  challenges  in  extending  the  influence  of  the 
Weapons  Systems  Acquisition  Reform  Act.  These  challenges  include: 
limited  organizational  capacity  to  support  cost  estimating,  performance 
assessment,  systems  engineering,  and  developmental  testing;  lack  of 
guidance  in  certain  areas;  limited  dissemination  of  lessons  learned  related 
to  systematic  problems  and  best  practices;  and  differences  between  the 
Office  of  the  Secretary  of  Defense  and  the  military  services  about  what 
constitutes  an  appropriate  level  of  risk  and  whether  the  benefits  of  certain 
reform  provisions  are  worth  the  cost 

As  systems  become  more  complex  and  rely  more  heavily  on  COTS  this  task 
becomes  more  difficult.  The  challenge  of  systems  engineering  (SE)  and  system  of 
systems  engineering  (SOSE)  is  clearly  articulating  the  terminology  in  the  context  of  each. 
Systems  engineers  may  opine  SOSE  is  merely  an  extension  of  SE  but  SOSE  practitioners 
will  point  out  there  are  key  differences.  Eor  example  application  of  the  SE  process 
includes  clearly  defining  system  requirements  at  the  beginning  of  a  program.  A  SOSE 
engineer  will  most  likely  be  looking  at  systems  which  already  have  established 
requirements.  Understanding  the  differences  in  approach  is  critical  during  each  phase  for 
developing  a  system  of  systems  such  as  CSRR.  This  section  looks  at  the  meaning  of 
systems,  system  of  systems,  systems  engineering,  and  system  of  systems  engineering  in 
order  to  provide  a  common  understanding. 

1.  Systems 

A  system  is  defined  by  INCOSE  as  “an  integrated  set  of  elements,  subsystems,  or 
assemblies  that  accomplish  a  defined  objective.  These  elements  include  products 
(hardware,  software,  firmware),  processes,  people,  information,  techniques,  facilities, 
services,  and  other  support  elements”  (Haskins  2014).  The  Systems  Engineering  Book  of 
Knowledge  (SEBOK)  vL3  (Adcock  2014)  defines  a  system  as  “a  set  of  elements  and  a  set 
of  inter-relationships  between  the  elements  such  that  they  form  a  bounded  whole  relative 
to  the  elements  around  them.”  Eurthermore,  the  SEBOK  vl.3  refers  to  Bertalanffy’s 
(Adcock  2014)  discussion  of  systems  within  his  general  systems  theory  as 
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General  system  theory  (GST),  attempts  to  formulate  prineiples  relevant  to 
all  open  systems.  GST  is  based  on  the  idea  that  eorrespondenee 
relationships  (homologies)  exist  between  systems  from  different 
diseiplines.  Thus,  knowledge  about  one  system  should  allow  us  to  reason 
about  other  systems. 

Other  definitions  inelude  the  Defense  Systems  Management  College’s  (DSMC) 
(2001)  view  “Simply  stated,  a  system  is  an  integrated  eomposite  of  people,  produets,  and 
proeesses  that  provide  a  eapability  to  satisfy  a  stated  need  or  objeetive.” 

The  NASA  System  Engineering  Guide  (2007,  3)  provides  a  slightly  different  if 
more  teehnieal  view  of  a  system  in  the  following  quote 

A  eonstruet  or  oolleetion  of  different  elements  that  together  produee 
results  not  obtainable  by  the  elements  alone.  The  elements,  or  parts,  ean 
inelude  people,  hardware,  software,  faeilities,  polieies,  and  doeuments; 
that  is,  all  things  required  to  produee  system-level  results.  The  results 
inelude  system-level  qualities,  properties,  eharaeteristies,  funetions, 
behavior,  and  performanee. 

The  formation  of  the  eomponents  or  elements  to  ereate  a  useful  produet  is  visible 
in  almost  everything  we  see,  hear,  or  toueh.  Nature  itself  has  systems  (e.g.  the  eeosystem) 
whieh  is  made  up  of  a  set  of  elements  to  ereate  a  forest,  or  desert  or  reef. 

2,  System  of  Systems 

There  is  a  great  deal  of  disagreement  over  what  is  the  definition  of  a  system  of 
systems  (SOS).  One  of  the  generally  aeeepted  definitions  has  been  defined  by  INCOSE. 
Aeeording  to  INCOSE  (Adcoek  2014)  defines  a  SOS  as 

systems-of-interest  whose  elements  are  themselves  systems;  typieally 
these  entail  large-scale  inter-disciplinary  problems  involving  multiple, 
heterogeneous,  distributed  systems.  These  interoperating  collections  of 
component  systems  usually  produce  results  unachievable  by  the  individual 
systems  alone 

The  Assistant  Secretary  of  the  Navy  for  Research,  Development  and  Acquisition 
(ASN  (RDA))  released  a  supplemental  guide  Systems  of  Systems  Engineering  Guidebook 
Version  2.0  (2006)  defining  system  of  systems  as  an 
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integrated  foree  package  of  interoperable  systems  acting  as  a  single  system 
to  achieve  a  mission  capability.  Typical  characteristics  include  a  high 
degree  of  collaboration  and  coordination,  flexible  addition  or  removal  of 
component  systems,  and  a  net-centric  architecture.  Individual  systems 
within  the  SOS  may  be  capable  of  independent  operations  and  are 
typically  independently  managed 

The  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  (USD 
(AT&L))  (Director  Systems  and  Software  Engineering  2008)  defined  a  system  of  systems 
as 


a  set  or  arrangement  of  systems  that  results  when  independent,  and  task- 
oriented  systems  are  integrated  into  a  larger  systems  construct,  that 
delivers  unique  capabilities  and  functions  in  support  of  missions  that 
cannot  be  achieved  by  individual  systems  alone 

Jamshidi  (2009),  in  his  opening  chapter  section  1.2,  outlines  six  definitions  of 
system  of  systems  from  other  researchers  (Jamshidi  2009,  3).  These  are  listed  in  Table  4 
below  and  illustrate  the  wide  disparity  in  determining  what  a  SOS  is  and  what  it  should 
be  able  to  do. 


Table  4.  System  of  Systems  Definitions  (from  Jamshidi  2009,  3) 


1. 

Enterprise  system  of  systems  engineering  (SOSE)  is  focused  on  coupling 
traditional  systems  engineering  activities  with  enterprise  activities  of  strategic 
planning  and  investment  analysis  [Carlock  and  Eenton,  2001]. 

2. 

System-of-systems  integration  is  a  method  to  pursue  development,  integration, 
interoperability,  and  optimization  of  systems  to  enhance  performance  in  future 
battlefield  scenarios  [Pei,  2000].  [Euskasik,  1998]. 

3. 

Systems  of  systems  exist  when  there  is  a  presence  of  a  majority  of  the 
following  five  characteristics:  operational  and  managerial  independence, 
geographic  distribution,  emergent  behavior,  and  evolutionary  development 
[Jamshidi,  2005]. 

4. 

Systems  of  systems  are  large-scale  concurrent  and  distributed  systems  that  are 
comprised  of  complex  systems  [Jamshidi,  2005;  Carlock  and  Eenton,  2001]. 

5. 

In  relation  to  joint  war-fighting,  system  of  systems  is  concerned  with 
interoperability  and  synergism  of  Command,  Control,  Computers, 
Communications,  and  Information  (C4I)  and  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  Systems  [Manthorpe,  1996]. 

6. 

SOSE  involves  the  integration  of  systems  into  systems  of  systems  that 
ultimately  contribute  to  evolution  of  the  social  infrastructure  [Euskasik,  1998] 
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The  following  quote  is  Jamshidi’s  definition  of  a  system  of  systems 


systems  of  systems  are  large-seale  integrated  systems  whieh  are 
heterogeneous  and  independently  operable  on  their  own,  but  are 
networked  together  for  a  common  goal.  The  goal,  as  mentioned  before, 
may  be  cost,  performance,  robustness,  etc.  (2009,  4) 

Vaneman  and  Budka  (2013,  2)  identified  a  system  of  systems  as  “SOS  as  a  system 
of  all  platforms,  assets,  systems,  nodes,  and  networks  that  join  together  to  achieve  a 
capability  needed  to  conduct  a  mission.”  These  definitions  would  be  applicable  to  many 
systems  within  the  DOD  inventory. 

The  contrast  of  this  however  is  discussed  by  Dahmann,  Rebovich  and  Lane  (2008) 
that  many  of  the  systems  within  DOD  were  created  as  standalone  systems,  failing  to 
benefit  from  the  consideration  of  how  they  will  fit  in  the  overall  defense  architecture. 
Table  5  identifies  the  differences  between  a  system  and  a  system  of  systems.  Systems 
engineers  and  SOS  engineers  must  work  to  synchronize  their  efforts  in  order  to  meet  the 
requirements  for  cost,  schedule,  and  performance  for  each. 

In  the  context  of  command  and  control  Manthorpe  (1996,  305-310)  stated  “Linking 
systems  into  joint  system  of  systems  allows  for  the  interoperability  and  synergism  of 
Command,  Control,  Computers,  Communications,  and  Information  (C4I)  and 
Intelligence,  Surveillance  and  Reconnaissance  (ISR)  System.”  Systems  with  joint 
requirements  such  as  the  Global  Command  and  Control  System-Maritime  (GCCS-M)  are 
becoming  increasingly  interlinked  in  terms  of  requirements,  development  and  acquisition. 
Their  ability  to  operate  only  with  similar  systems  within  their  networks  prevents  effective 
employment  of  information  supporting  the  user.  The  increasing  complexity  and  use  of 
networks  to  support  war  fighting  has  made  the  requirement  to  develop  systems  that  work 
together  increasingly  visible.  The  holistic  view  provided  by  a  SOS  approach  provides  a 
more  clearly  defined  role  of  how  a  system  will  fit  in  the  overall  SOS  architecture.  In 
summary  a  SOS  capability  is  greater  than  the  sum  total  of  the  individual  components. 
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Table  5.  Comparison  of  Systems  and  System  of  Systems  (from  Dahmann, 

Rebovieh  and  Lane  2008,  5) 


Aspect  of 
Environment 

System 

Acknowledged  SoS 

Management  and  Oversight 

Stakeholder 

Involvement 

Clearer  set  of  stakeholders. 

Stakeholders  at  both  system  level  and  SoS  levels  (including  the  system 
owners,  with  competing  interests  and  priorities):  in  some  cases,  the  system 
stakeholder  has  no  vested  interest  in  the  SoS;  all  stakeholders  may  not  be 
recognized. 

Governance 

Aligned  project  manager  and 
funding. 

Added  levels  of  complexity  due  to  management  and  funding  for  both  the  SoS 
and  Individual  systems;  SoS  does  not  have  authority  over  all  the  systems. 

Operational  Environment 

Operational 

Focus 

Designed  and  developed  to  meet 
operational  objectives. 

Called  upon  to  meet  a  set  of  operational  objectives  using  systems  whose 
objectives  may  or  may  not  align  with  the  SoS  objectives. 

Implementation 

Acquisition 

Aligned  to  acquisition  categories 
milestones,  documented 
requirements,  SE  with  a 

SE  plan. 

Added  complexity  due  to  multiple  system  lifecycles  across  acquisition 
programs,  involving  legacy  systems,  developmental  systems,  new 
developments,  and  technology  insertion;  typically  have  stated  capability 
objectives  up  front  which  may  need  to  be  translated  into  formal  requirements. 

Test  and 
Evaluation 

Test  and  evaluation  of  the 
system  is  generally  possible. 

Testing  is  more  challenging  due  to  the  difficulty  of  synchronizing  across 
multiple  systems'  life  cycles,  given  the  complexity  of  all  the  moving  parts  and 
potential  for  unintended  consequences. 

Engineering  and  Design  Considerations 

Boundaries 

and 

Interfaces 

Focuses  on  boundaries  and 
interfaces  for  the  single  system. 

Focus  on  identifying  the  systems  that  contribute  to  the  SoS  objectives  and 
enabling  the  flow  of  djata,  control,  and  functionality  across  the  SoS  while 
balancing  needs  of  the  systems. 

Performance 
and  Behavior 

Performance  of  the  system  to 
meet  specified  objectives. 

Performance  across  the  SoS  that  satisfies  SoS  user  capability  needs  while 
balancing  needs  of  the  systems. 

Systems  of  systems  are  elassified  as  one  of  four  types  (Direetor,  Systems  and 
Software  Engineering  2008;  ASN  (RDA)  2006;  Vaneman  &  Budka  2013,  3):  virtual, 
collaborative,  acknowledged  and  directed.  The  type  of  SOS  to  be  used  can  be  planned 
from  the  beginning  (directed)  such  as  an  entire  architecture  such  as  the  littoral  combat 
ship  (LCS),  ORP,  or  GPS.  A  virtual  SOS  is  an  ad  hoc  grouping  of  systems  into  a  largely 
cooperative  effort.  A  good  example  may  be  the  organization  of  units  in  response  to  an 
emergency.  A  collaborative  SOS  example  is  the  local  internet  service  provider  or  the 
Defense  Acquisition  Community  Connection  Communities  of  Practice.  The 
acknowledged  SOS  is  the  most  common  form  seen  across  DOD.  This  can  be  seen  when  a 
cooperative  agreement  exists  between  individual  participants  and  the  lead  systems 
integrator  role.  Each  type  is  described  in  Table  6. 
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Table  6.  System  of  Systems  Types  (from  Director,  Systems  and  Software 

Engineering  2008,  4-5) 


Types  of  System 
of  Systems 

Description 

Virtual 

A  virtual  SOS  is  essentially  an  ad  hoc  group  of  systems.  There  is 
no  single  authority  or  agreed  upon  purpose.  Large  scale  behavior 
emerges  and  may  be  desirable.  But  this  type  of  SOS  must  rely 
upon  invisible  mechanisms  to  maintain  it. 

Collaborative 

Component  systems  interact  more  or  less  voluntarily  to  fulfill 
agreed  upon  central  purposes.  Stakeholders  work  collectively  to 
provide  some  means  to  enforce  and  maintain  standards  (such  as 
interface  standards). 

Acknowledged 

There  are  recognized  objectives,  a  designated  manager,  and 
resources  for  the  SOS.  Constituent  systems  retain  their 
independent  ownership,  objectives,  funding,  and  development 
and  sustainment  approaches.  Changes  in  the  systems  are  based 
on  collaboration  between  the  SOS  and  the  system.  A  lead 
systems  integrator  may  be  an  acknowledged  SOS. 

Directed 

The  integrated  system-of-sys terns  is  built  and  managed  to  fulfill 
specific  purposes.  It  is  centrally  managed  during  long-term 
operation  to  continue  to  fulfill  those  purposes  as  well  as  any  new 
ones  the  system  owners  might  wish  to  address.  The  component 
systems  maintain  an  ability  to  operate  independently,  but  their 
normal  operational  mode  is  subordinated  to  the  central  managed 
purpose.  An  example  of  this  would  the  USS  Virginia  submarine 
program.  The  platform  is  made  up  of  numerous  smaller  systems 
and  system  of  systems. 

A  SOS  in  contrast  to  a  system,  however  complex,  will  evolve  over  time  as 
functions,  components,  and  requirements  change  for  the  individual  systems  (Jamshidi 
2009).  Managing  a  SOS  presents  additional  challenges  due  to  the  nature  of  the  evolution, 
or  fuzziness  of  future  requirements.  The  SOS  engineer  not  only  must  consider  the 
individual  systems  capabilities  and  requirements  but  must  arrange  all  of  the  components 
in  an  optimal  manner  to  achieve  operational  synergy. 

Redundancy  within  a  SOS  is  a  factor  which  can  separate  it  from  a  system. 
Redundancy  can  have  different  meanings  but  typically  becomes  a  question  of  the  risk  of 
failure  versus  the  consequences  of  failure.  Traditional  and  sociotechnical  systems  can  be 
categorized  as  Type  I  or  Type  II  (Jamshidi  2009,  199).  Type  I  SOS  have  redundancy  built 
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in;  if  one  system  should  fail  the  others  are  capable  of  reconfiguring  or  rerouting  to 
continue  operations.  Type  II  SOS  have  no  such  redundancy. 

Resilience  is  another  characteristic  of  a  SOS.  Jackson  and  Ferris  (2013)  outlined  in 
their  paper  the  principles  of  resilience  in  an  engineered  system.  Using  the  definition 
recognized  by  the  U.S.  government  resilience  is  “the  ability  to  adapt  to  changing 
conditions  and  prepare  for,  withstand,  and  rapidly  recover  from  disruption”  (Jackson  and 
Ferris  2013,  153).  They  go  into  further  detail  to  break  down  this  definition  and  define  the 
14  principles  for  resilience. 

3,  Systems  Engineering 

Systems  engineering  (SE)  dates  back  to  the  1940s  to  the  Bell  laboratories.  The 
actual  usage  of  SE  principles  dates  earlier  to  the  1900s  (Buede  2000,  6).  Engineers  and 
scientists  were  applying  the  principles  within  their  own  specialties  but  as  they  crossed 
with  other  fields  new  rules  for  developing  and  managing  a  system  were  required.  An 
early  example  of  systems  engineering  would  be  a  locomotive.  Whereas  a  formal 
requirements  review  probably  did  not  occur  the  information  still  had  to  be  defined.  The 
use  of  SE  identified  key  factors  such  as  size,  speed,  range  and  cost. 

INCOSE  defines  SE  as  “an  interdisciplinary  approach  and  means  to  enable  the 
realization  of  successful  systems”  (Haskins  2012,  7).  According  to  the  SEBOK  (Adcock 
2014)  SE 

focuses  on  holistically  and  concurrently  understanding  stakeholder  needs; 
exploring  opportunities;  documenting  requirements;  and  synthesizing, 
verifying,  validating,  and  evolving  solutions  while  considering  the 
complete  problem,  from  system  concept  exploration  through  system 
disposal 

SE  is  supported  by  technical  and  technical  management  processes  to  enable 
focusing  on  successful  development,  delivery,  and  sustainment  of  a  system  meeting  the 
customer’s  needs  while  considering  business  and  technical  needs  involved  (Haskins 
2012).  The  NASA  Systems  Engineering  Handbook  (NASA  2007,  3)  provides  a  more 
holistic  view: 
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Systems  engineering  is  the  art  and  seience  of  developing  an  operable 
system  capable  of  meeting  requirements  within  often  opposed  constraints. 
Systems  engineering  is  a  holistic,  integrative  discipline,  wherein  the 
contributions  of  structural  engineers,  electrical  engineers,  mechanism 
designers,  power  engineers,  human  factors  engineers,  and  many  more 
disciplines  are  evaluated  and  balanced,  one  against  another,  to  produce  a 
coherent  whole  that  is  not  dominated  by  the  perspective  of  a  single 
discipline 

SE  processes  within  various  organizations  share  many  common  traits.  A  study 
funded  by  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
(USD  (AT&L))  (Patterson,  Dub  in,  and  Richter  2004)  evaluated  the  SE  activities 
performed  by  each  service,  civilian  government  agencies,  associations,  the  models 
available  and  education  opportunities.  Understanding  these  activities  provides  a  common 
framework  for  effective  use  of  SE  processes  and  principles  used  by  many  programs  and 
projects. 

4.  System  of  Systems  Engineering 

Using  Wikipedia  as  a  starting  point,  system  of  systems  engineering  (SOSE)  is  a 
set  of  developing  processes,  tools,  and  methods  for  designing,  re-designing  and  deploying 
solutions  to  system-of-systems  challenges  {Wikipedia  2013).  DOD  develops  many 
complex  systems  which  require  using  SOSE  approaches.  A  more  scholarly  definition 
defines  “SOSE  involves  the  integration  of  systems  into  systems  of  systems  that  ultimately 
contribute  to  evolution  of  the  social  infrastructure  (Luskasik  1998  quoted  in  Jamshidi 
2009,  3).  The  Defense  Acquisition  Guidebook  (DAG)  (Defense  Acquisition  University 
[DAU]  2008)  defines  SOSE  as  “a  set  or  arrangement  of  systems  that  results  when 
independent  and  useful  systems  are  integrated  into  a  larger  system  that  delivers  unique 
capabilities.”  The  USD  (AT&L)  developed  their  initial  version  of  the  “Systems 
Engineering  Guide  for  System  of  Systems  ”  (Director,  Software  and  Systems  Engineering 
2006)  and  ran  a  pilot  to  evaluate  18  programs  within  DOD.  They  found  the  following: 

1.  Systems  of  systems  tend  to  be  continual  efforts  addressing  user  needs 
through  a  combination  of  systems. 

2.  Systems  of  systems  typically  are  not  new  acquisition  programs. 

3.  Program  managers  of  a  system  of  systems  usually  do  not  have  requirements 
or  funding  for  the  individual  systems. 
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4.  System  of  systems  capabilities  evolve  over  time. 

5.  A  well  designed  system  of  systems  is  more  capable  of  incorporating 
incremental  upgrades. 

USD  (AT&L)  directed  an  update  to  its  Systems  Engineering  Guide  for  System  of 
Systems  (USD  SOSE  Guide)  V.9  to  Vl.O  recognizing  the  value  SOSE  has  in  enabling  the 
development  of  complex  systems  in  today’s  environment.  The  USD  SOSE  Guide  defines 
SOSE  as  “planning,  analyzing,  organizing,  and  integrating  the  capabilities  of  a  mix  of 
existing  and  new  systems  into  an  SOS  capability  greater  than  the  sum  of  the  capabilities 
of  the  constituent  part”  (Director,  Systems  and  Software  Engineering  2008).  Additionally 
USD  (AT&E)  recognized  SOSE  must  consider  a  variety  of  systems  whether  they  are 
fully  developed,  or  in  an  as  yet  to  be  defined  state(Director,  Systems  and  Software 
Engineering  2008)  . 

The  challenge  of  SOSE  varies  due  to  the  level  of  complexity  involved.  This  can 
vary  from  spontaneous,  short-lived,  simple  SOS  to  long  lived,  complex,  and  continuously 
evolving  SOS  (Jamshidi  2009).  Examples  of  a  simple,  ad  hoc  SOS  may  be  a  planned 
response  to  a  casualty  or  disaster.  In  this  case  each  individual  system  (e.g.,  the  police, 
fire,  paramedics,  utilities,  and  other  groups)  each  provide  a  component  for  responding  to 
the  event.  Individually,  they  are  still  capable  of  accomplishing  their  primary  role,  but 
collectively  they  comprise  a  larger  capability  (e.g.,  responding  to  a  mass  casualty).  A 
more  complex  example  defined  by  Jamshidi  (2009,  15)  is  a  “galactic  SOS”  which  is 
represented  by  an  ecosystem  or  a  community.  Jamshidi  continues  by  stating  these 
complex  SOS  are  characterized  with 

an  open  systems  approach  to  create  a  healthy,  dynamic  architecture, 
enabling  them  to  effectively  capitalize  on  open  systems  development 
principles  and  strategies  such  as  modular  design,  standardized  interfaces, 
emergence,  natural  selection,  conservation,  synergism,  symbiosis, 
homeostasis,  and  self-organization  (Jamshidi  2009,  16) 

An  additional  challenge  to  SOSE  is  the  lead  engineer  must  routinely  integrate 
existing  systems.  Since  they  do  not  control  the  requirements,  boundaries  and 
development,  the  end  result  can  be  suboptimal.  External  environmental  effects  further 
complicate  issues  for  the  lead  engineer.  The  complexity  of  engineering  a  SOS  using  the 
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trapeze  model  can  be  improved  through  implementing  a  wave  model  (Dahmann  et  al. 
2011).  The  colored  blocks  shown  of  the  trapeze  model  in  Figure  17  represent  the 
different  activities  and  how  they  are  related  to  each  other.  The  activities  move  from 
translating  objectives  to  assessing  performance  and  understanding  the  systems  to 
evolving  the  SOS  architecture.  For  example  attempting  to  orchestrate  upgrades  requires 
the  SOS  engineer  understand  how  the  systems  are  interrelated,  what  the  capability 
objectives  are  in  terms  of  the  individual  systems  and  the  overarching  SOS.  The  arrows 
between  the  SOS  engineering  activities  swing  back  and  forth  like  a  trapeze.  This  in  turn 
drives  how  to  assess  the  performance  impacts  of  the  proposed  upgrades.  The  drawback  of 
the  trapeze  model  is  the  sequence  of  activities  is  not  clearly  defined  and  hard  to  follow. 

The  sequence  of  events  within  the  trapeze  model  can  be  transformed  by 
“unwrapping”  the  process  and  correlating  the  processes  to  the  wave  model.  The  colored 
blocks  are  aligned  on  the  left  within  a  swim  lane  to  show  where  they  would  occur. 
Converting  to  a  wave  model  provides  a  better  detail  of  the  activities  and  their  sequence 
(Dahmann  et  al,  2011).  Figure  18  illustrates  the  unwrapping  and  final  results  as  a  wave 
model.  The  unwrapped  wave  model  provides  a  much  clearer  picture  of  the  order  of  the 
activities  for  analysis,  development  and  implementation. 


External  Environment 


Figure  17.  Trapeze  Model  (from  Dahmann  et  al  201 1,  2) 
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Trapeze  Model 


Figure  18.  Unwrapping  the  Trapeze  Model  into  the  Wave  Model  (from 

Dahmann  et  al  201 1,  3) 


Another  factor  challenging  SOSE  is  the  lack  of  a  standardized  language. 
Thousands  of  standards  are  developed  by  hundreds  of  organizations  which  contribute  to 
the  confusion  when  discussing  SOSE  and  comparing  one  against  another.  These 
standards  and  their  application  can  be  derived  into  a  concept  of  “universally  agreed-upon 
set  of  guidelines  for  interoperability”  (Jamshidi  2009,  457).  The  guidelines  define  the 
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following  levels  of  standardization:  compatibility,  interchangeability,  commonality,  and 
reference.  From  an  SOS  perspective  these  levels  create  “compatibility,  similarity, 
measurement,  and  symbol  and  ontological  standardization”  (Johnson  2009,  457).  As  SOS 
development  evolves  and  matures  standards  among  the  various  disciplines  will  be 
created.  Johnson  (2009,  461)  emphasizes  without  a  common  language  a  SOS  cannot 
communicate  with  other  SOSs,  will  not  be  fully  functional,  and  is  not  capable  of  allowing 
new  components  to  be  added  without  significant  effort. 

A  system  of  systems  needs  a  means  of  governance.  Governance  encompasses  all 
of  the  processes  to  create  and  manage  an  organization  regardless  if  it  is  formal  or 
informal.  Effective  governance  will  define  the  activities  that  need  to  occur,  identify  who 
has  authority  to  accomplish  those  actions  and  verify  they  are  being  accomplished. 
Without  effective  governance,  similar  to  a  lack  of  effective  C2,  the  risks  of  delays  or 
failure  increase  greatly.  Challenges  to  successful  SOS  governance  were  identified  by  a 
study  performed  by  the  Center  for  Public  Policy  and  Private  Enterprise  (Gansler, 
Eucyshyn  and  Rigilano  2012,  viii).  Eive  categories  were  identified  which  can  affect 
successful  SOS  governance:  leadership,  management,  requirements,  human  resources  and 
funding  (Gansler,  Eucyshyn  and  Rigilano  ix-xiii).  Berteau  et  al  (2013,  646)  presented 
eight  attributes  for  the  acquisition  governance  of  a  SOS  using  the  future  combat  system 
(ECS)  and  maritime  domain  awareness  programs  (MDA)  as  case  studies  shown  in  Eigure 
19.  The  Software  Engineering  Institute  performed  a  study  on  system  of  systems  attributes 
to  define  the  characteristics  of  effective  governance.  They  identified  six  characteristics 
concerning  collaboration,  accountability,  evolution,  and  processes  (Morris,  Place  and 
Smith  2006). 

The  Deputy  Assistant  Secretary  of  the  Navy  for  Research,  Development,  Testing 
and  Evaluation  (DASN(RDT&E)  Chief  Engineer  and  SPAWAR  Information  Technology 
Technical  Authority  (ITTA)  identified  five  SOS  qualities  necessary  for  good  governance 
(Vaneman  and  Jaskot  2013).  Table  7  describes  the  characteristics  of  the  directed  and 
virtual  SOS  with  collaborative  and  acknowledged  SOS  somewhere  in  between. 
Application  of  these  principles  provides  a  framework  amenable  to  managing  the  activities 
surrounding  system  of  systems  development  and  sustainment. 
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GOVERNANCE  ATTKI6UTE  DESCRIPTION 

LEVEL  OF 

ORGANIZATIONAL  FOCUS 

Tbe  Irv'cl  il  utiicb  SoS  g<n'(TiuDC«  (xcun  aithin  Ibr  offjnnuQon.  Tbis  is  not  tb< 
samf  as  ss-sinns'capabdOKs  focus  or  trchnicaJ  focus,  both  ofuhicb  arc  oursxlc  tbe 
scope  of  tbe  CSIS  SoS  governance  aiiahsis 

INTEGRATION  OF 

FUNCTIONAL  END-USER 

NEEDS 

Tbe  mechamsnHs;  and  frequency  uitb  «1ucb  tbe  fuDctional  needs  of  end-users  are 
buih  inio  tbe  systeni-of-systems,  and  at  sshicb  points  in  tbe  process  of  delb  ering  tbe 
system-of-systents  this  incorporabon  occurs. 

DECISION  MAKING 

AUTHORITY 

Tbe  formal  mecbaiusm  by  wbicb  debvery  of  syitems-of-systems  is  gcAeroed, 
including  bow  budget  is  allocated,  standards  are  set.  tredeofTs  are  managed,  and 
inconsistencies  are  atl^udkaled. 

ENFORCEMENT 

Tbe  mechanisms  and  lesel  of  management  oversight  by  which  tbe  objectives  of  the 
SoScapabibly  to  be  dehs'errd  are  ensured,  mctudins  measurements  and  program 
metises. 

WORKFORCE 

The  ejtaminatson  of  SoS  w  orkforce  stroemres.  unity  of  mission,  and  capabity 
development  and  enhancemenl  through  use  of  contracted  support. 

•M 

INCENTIVE  STRUCTURE 

Tbe  alignment  betw  een  enterprise  goab  and  tbe  incentive  and  reward  structures 
of  the  components  that  onplnnent  them 

KNOWLEDGE  OWNERSHIP 
/ACCESS  TO  KNOWLEDGE 

The  accessibility'  of  information  regarding  tbe  operating  environment,  technical 

staodards.  and  odier  dramits  tbai  compilse  the  miniMf'syytniu 

RISK  ASSESSMENT  /  RISK 
MANAGEMENT 

Risk  assessments  and  management  strategies  tailored  to  mission  accooiphshmeni 
and  the  flexibiity  and  resdience  required  for  debs'emg  systems-of-systems  in  tbe 
face  of  unforeseen  developroents. 

Figure  19.  Attributes  of  Acquisition  Governance  (from  Berteau  et  al  2013,  14) 


Testing  and  evaluation  of  a  SOS  involves  taking  a  broader,  more  holistic 
perspective.  Brooks  and  Sage  (2005/2006,  268)  pointed  out  while  SOS  integration  is  a 
complex  affair  the  ability  to  accomplish  validation  and  verification  are  just  as  important. 
The  FY2000  DOT&E  (2000,  section  lV-170)  annual  report  pointed  out  several  important 
points  regarding  testing  in  terms  of  challenges  and  benefits. 

1 .  Shorter  acquisition  cycles  due  to  accelerations  to  field  technology  would 
make  testing  more  challenging. 

2.  The  use  of  COTS  should  be  tempered  with  the  risks  to  degrading  training, 
logistics,  and  documentation. 

3.  Effective  use  of  the  requirements  and  testing  strategy  can  keep  the  program 
focused  on  the  goal. 
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Table  7.  System  of  System  Characteristies  (from  Vaneman  and  Jaskot  2013) 


Characteristics 

Definition 

Directed  SOS 

Virtual  SOS 

Autonomy 

The  ability  to  make 
independent  choices; 
the  right  to  pursue 
reasons  for  being  and 
fulfilling  purposes 
through  behaviors 

Conformance:  Autonomy 
is  ceded  by  parts  in  order 
to  grant  autonomy  to  the 
system 

Independence:  Autonomy 
is  exercised  by  constituent 
system  in  the  order  to 
fulfill  the  purpose  of  the 
SOS. 

Belonging 

To  be  a  member  of  a 
group;  to  have  the 
proper  qualifications 

Centralize:  To  bring 
under  one  control;  to 
come  together  to  form  a 
center. 

Decentralization: 
Constituent  systems 
choose  to  belong  on  a 
cost/benefit  basis,  also  in 
order  to  cause  greater 
fulfillment  of  their  own 
purposes,  and  because  of 
belief  in  the  SOS  supra 
purpose. 

Connectivity 

The  ability  of  a  system 
to  link  with  other 
systems. 

Platform-centric- 
Prescient  design,  along 
with  parts,  with  high- 
connectivity  among  sub¬ 
systems. 

Network-centric; 
Dynamically  supplied  by 
constituent  systems  with 
every  possibility  of 
myriad  connections 
between  constituent 
systems,  possible  via  a 
network-centric 
architecture  to  enhance 

SOS  capability. 

Diversity 

Noticeable 

heterogeneity,  having 
distinct  or  unlike 
elements  or  qualities 
in  a  group;  the 
variation  of  social  and 
cultural  identities 
among  people  existing 
together  in  an 
operational  setting. 

Homogeneous:  Managed, 
that  is,  reduced  or 
minimized  by  modular 
hierarchy;  parts  diversity 
encapsulated  to  create  a 
known  discrete  module 
whose  nature  is  to  project 
simplicity  into  the  next 
level  of  hierarchy. 

Heterogeneous:  Increased 
diversity  in  SOS 
capability  achieved  by 
released  autonomy, 
committed  belonging,  and 
open  connectivity. 
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Characteristics 

Definition 

Directed  SOS 

Virtual  SOS 

Emergence 

The  appearance  of 
new  properties  in  the 
course  of  development 
or  evaluation. 

Foreseen:  Foreseen,  both 
good  and  bad  behavior, 
and  designed  in  or  tested 
out  as  appropriate. 

Indeterminable:  Enhanced 
by  deliberately  not  being 
foreseen,  though  its 
cmcial  importance  is,  and 
by  creating  and 
emergence  capability 
climate,  that  will  support 
early  detection  and 
elimination  of  bad 
behaviors. 

F.  CASE  STUDIES 

Case  studies  provide  the  opportunity  to  eapture  a  teaehable  moment,  learning 
prineiple,  a  lesson  learned,  and  share  the  results  with  others.  Case  studies  are  an  effeetive 
means  to  train  engineering  and  aequisition  professionals  about  real  systems  in  use  today 
(Soy  1997;  Bahill  and  Chapman  1994,  145).  Stjelja  (2013,  3)  made  the  following 
eomment  about  ease  studies  “An  important  aim  of  the  ease  study  approaeh  is  to  eapture 
the  complexity  of  a  single  case.”  Additionally  Stjelja  continues  “Case  studies,  thus, 
cannot  be  defined  through  its  research  methods  but  rather  through  an  interest  in  what  is  to 
be  studied,  and  given  the  definition  above  it  should  be  noted  that  a  case  study  is  not  a 
method  but  a  research  strategy  (Stjelja  2013,  3). 

A  case  study  can  take  a  number  of  approaches.  Soy  (1997)  uses  a  methodology 
with  six  steps  from  defining  the  questions  to  completing  the  final  report.  Bahill  and 
Chapman  (1994)  outline  a  commercial  approach  of  benefits  versus  efforts.  The  Friedman 
and  Sage  framework  (2003,  88-96)  shown  in  Figure  20  and  Table  8  analyzes  nine  areas 
and  stratifies  responsibilities  by  contractor,  government,  and  shared  responsibilities. 
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Figure  20.  Friedman  and  Sage  Framework  (from  Friedman  and  Sage  2003,  88) 
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Table  8.  Description  of  the  Friedman  and  Sage  Framework  Domains  (after 

Friedman  and  Sage  2003,  89-96) 


A 

B 

c 

Integration  approach 

Prime  contractor  is  responsible 

forSOSE 

Govt  acts  as  integrator  and 

program  manager 

Govt  contracts  out  individual 

components  and  theSOSE 
responsibilities 

A 

Requirements  definition  and 
management 

Requirements  shall  flow 

downward  in  a  coherent  and 

traceable  manner  from  the  top 

1  evel  to  a  1 1  1  ower  1  evel  s 

Customer  and  contractor  shall 

share  their  knowledge  of  the 
technical  maturity  relative  to 
new,  unprecedented  systems 
being  engineered 

Govtshall  integrate  the  needs  of 
its  user  organizations  woth  he 
management  activities  of  its 
engineering  organizations 

B 

Systems  architecture 
development 

The  system  baseline  architecture 
shall  be  established  early  in  the 
program  and  involve  all 

dimensions  of  technical  issues  as 

well  as  customer  needs  and 
satisfaction,  political  pressures 
and  continuity  of  funding 

The  systems  architecture  should 
be  established  early  and  the  best 
judgement  of  the  government  and 
contractor  shall  be  employed  on 
all  key  issues  including  use  of 
new  or  legacy  systems 

Atotal  systems  architectureshall 
be  established  early  in  ordero  to 
provide  a  sound  basis  of 

effectiveness  across  the  broadest 

spectrum  of  contractors  and 
operations 

C 

System/subsystem  design 

System  desing  shall  proceed  in  a 
logical  and  orderly  manner 
through  a  process  of  functional 
decomposition  and  design 
traceability  that  originates  with 
the  system  functional  architecture 
and  results  in  design 
specifications  for  the  system 

The  government  customers  and 

contractors  shall  share  the 

systems  design  responsbility 

The  user  shall  share  measures  of 

effectiveness  to  ensure  the 

proposals  selected  are  those 
most  responsivie  to  all 
stakeholders,  especially 
operational  organizations 

D 

Systems  Integration  and 

interfaces 

The  contractor  shal  1  assure  the 

systems  integration  and 
interfaces  ateach  level  suipports 
total  system  functionality  across 
the  lifel  cycle 

The  contractor  and  government 
shall  assureall  systems  are 
integrated  within  themselves  as 
well  as  interfaced  with  existing 
systems 

Thegovernmentshall  assure  that 
all  operational  systems  in 
planning,  development  or 
deployed  are  compatible  and 
mutually  supportive  in  a  broad 
system  of  systems  or  family  of 
systems  context 

E 

Verification/validation 

Every  requirementshall  havea 
test  and  every  test  shall  have  a 
requirement 

Government  facilities  shall  be 

shared  to  perform  V&V  and  the 
test  criteria  shall  be  shared  early 

Thegovernmentshall  have  the 

final  word  on  the  confidence 

levels  derived  from  the  testing 
during  development,  operational 

test  and  evaluation  and  actual 
deployment  and  operational  use 

F 

Deployment  and  post 
deployment 

The  contractor  shall  maintain  the 
appropriate  engineering  and 
testing  capabilities  to  support 
gathering,  analyzing,  and 
recommending  possible  changes 
to  the  system  design  or  support 
through  reengineering 

The  government  and  contractor 
shall  cooperate  to  conduct  an 
OPEVAL  and  be  open  to  feeding 
information  backto  the  program 
managers  to  consider  design 
changes  or  modifications  through 
reengineering 

Thegovernmentshall  ensurea 
proper  OPEVAL  occurs  and  that 
all  data  gathered  from  the  tests 
be  eva  1  a  uted  for  potent!  a  1 
recommendations  for  system 
modification,  redesign,  or 
reengineering.  Pre  planned 
improvements  are  encouraged 

G 

Life  cycle  support 

All  design  activities  shall  be 
viewed  from  an  entire  life  cycle 
perspective 

A  balance  of  methods, 
measurements,  technologies  and 
processes  shall  be  employed  to 
support  the  entire  life  cycle 

Funding  support  across  the  life 
cycle  shall  be  maintained  and 
ea  rl  y  devel  opment  progra  ms 
shall  recognize  the  importance  of 
controlling  total  ownership  cost 

H 

Risk  assessment  and 

management 

Risks  shall  be  identified, 
prioritized  and  mitigated  at  all 
levels.  Risk  is  associted  with  cost, 

schedule  and  technical 

dimensions 

The  government  shall  ensure  risk 
management  is  part  of  the 
contractors  systems  engineering 
management  plan  (SEMP)  and 
risks  are  presented  at  all 
program  reviews 

Risk  management  at  all  levels 
shall  be  an  essential  and  inherent 

part  of  systems  engineering, 
program  planning  and  life  cycle 

activities 

1 

System  and  program 
management 

Every  program  shall  have  a  SEMP 
tailored  to  that  program.  The 
contractor  shal  1  also  develop  a 
tea  m  of  ca  pa  bl  e  systems 
engineers 

Systems  engineeringshall  be 
recognized  and  supported 
throughout  progam  development 
and  management 

Thegovernmentshall  establish 
security  levels  for  programs  to 
protectcrucial  technologies  and 
special  capabilities 
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Using  the  framework  enables  the  researeher  to  follow  a  eommon  approaeh  to 
examine  the  details  of  a  program.  Understanding  the  sueeesses  and  failures  of  a  program 
reveals  the  benefits  and  penalties  they  ineurred  whieh  ean  be  leveraged  to  avoid  repeating 
the  same  mistakes  and  identify  similar  opportunities  for  sueeess.  The  Secretary  of  the  Air 
Force  directed  several  initiatives  to  revitalize  the  systems  engineering  community  using 
case  studies  of  programs  within  the  Air  Force  and  NASA.  The  Air  Force  Institute  of 
Technology  and  NASA  wrote  a  series  of  systems  engineering  case  studies  about  the  A- 10 
Thunderbolt  (Jacques  and  Strouble  2010),  F-111  (Richey  2005),  Global  Positioning 
System  (O’Brien  and  Griffin  2007),  Hubble  Space  Telescope  (Mattice  2003), 
International  Space  Station  (Stockman,  Boyle,  and  Bacon  2010),  B-2  bomber  (Griffin  and 
Kinnu  2007),  C-5  Galaxy  (Griffin  2004),  Theater  Battle  Management  Core  System 
(TBMCS)  (Collens  and  Krause  2005),  KC-135  Aircrew  Training  System  (Chislaghi, 
Dyer  and  Free  2010),  Global  Hawk  (Kinzig  2010)  and  the  Miniature  Seeker  Technology 
Integration  (MSTI)  (Grenville,  Kleiner,  and  Newcomb  2004).  Each  case  study  identified 
“learning  principles”  capturing  significant  aspects  across  the  systems  engineering,  risk 
management  and  program  management  support  activities.  Most  of  the  case  studies 
implemented  a  Friedman  and  Sage  framework  to  capture  the  learning  principles. 
Examples  of  some  of  the  learning  principles  captured  are  listed  below  in  Table  9. 

NASA  acknowledged  the  value  of  case  studies.  The  Challenger  and  Columbia 
shuttle  accidents,  errors  with  the  Hubble  telescope,  oversight  from  Congress  and  the  fact 
two-thirds  of  their  workforce  is  approaching  retirement  provides  strong  motivation  to 
avoid  repeating  mistakes  and  capture  opportunities  for  improvement.  NASA  has  over  50 
case  studies  to  capture  learning  principles.  NASAs  GSEC  Case  Study  Methodology 
(NASA  2011,  3-8)  outlines  their  approach  for  an  effective  case  study.  These  steps  are: 

1 .  Pick  a  target,  preferably  something  which  will  offer  insight  to  others 

2.  Define  the  parameters  of  the  case 

3.  Do  the  homework  and  background  research 

4.  Interview  the  key  players  to  get  their  story 

5.  Evaluate  the  story  lines  for  learning  points 

6.  Draft  the  case  into  a  narrative 
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7.  Circulate  the  draft 

8.  Test  the  case  with  a  local  audience 

9.  Create  a  teaching  note  an  epilogue 

10.  Validate,  publish  and  roll  out  the  case 


Table  9.  Examples  of  Case  Study  Learning  Principles 


Case  Study 

Learning  Principles  (LP) 

F-111 

Riehey  (2005,  6)  identified  the  following  learning  principles: 

LPl-  Requirements  definition  and  management  were  poorly  coneeived 
and  diffieult  to  achieve.  The  attendant  speeifieations  made  the  F-111 
system  development  extremely  costly,  risky  and  difficult  to  manage. 

LP2-  Systems  architeeture  and  design  trade-offs-  System  engineering 
managers  were  not  allowed  to  make  important  tradeoffs  needed  in  order 
to  achieve  a  design  balanced  for  performance,  cost  and  mission 
effectiveness  and  the  related  risk  and  sehedule  impacts. 

LP3-  Communications  and  systems  management-  Poor  communications 
between  Air  Foree  and  Navy  staffs  and  over  management  by  the 
Seeretary  of  Defense  and  Congress  prevented  the  System  Program 
Offiee  direetor  from  applying  sound  systems  engineering  principles. 

LP4-  Validation  and  Verification-  Areas  of  risk  and  deficiencies  were 
discovered  during  RDT&E  even  though  there  was  pereeived  low  risk  in 
the  design. 

LP5-  Program  Management-  Cancellation  of  the  Navy’s  participation  in 
the  F-111  program  eame  after  the  design  was  frozen  causing  enduring 
impaets  on  the  Air  Force  F-111  performance  and  cost. 

TBMCS 

Cohens  and  Krause  (2005,  6)  identified  the  following  LP 

LP  1-  The  requirements  process  for  producing  the  first  release  of 
TBMCS  was  broken.  The  user  and  aequisition  communities  never  were 
on  the  same  page.  The  users  did  not  write  a  CONOPS  or  update  the 
original  ORD.  The  aequisition  community  wrote  a  teehnical 
requirements  document  for  the  contraetor  to  develop  their  system-level 
speeifieations.  None  of  these  doeuments  were  aligned  causing  further 
eonfusion. 

B-2 

Griffin  and  Kinnu  (2007,  53)  identified  this  LP. 

LP  4-  Subsystem  Maturity-  Identification  that  a  number  of  aeronautical 
systems  did  not  meet  the  performanee  baseline  eonfiguration  foreed  a 
reeonfiguration  of  a  number  of  subsystems  resulting  in  a  1 8  month 
schedule  slip. 
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The  SPAWAR  chief  engineers  office  developed  a  number  of  SE  and  SOSE 
studies  including  their  Information  Dominance  Enterprise  Architecture  (IDEA)  which 
seeks  to  outline  the  way  forward  for  achieving  mission  assurance.  While  these  are  not 
case  studies  these  documents  reflect  the  growing  role  of  SOSE  in  integrating  and 
delivering  capabilities  to  the  warfighter.  The  Navy  Capability  Evolution  Process  (NCEP) 
(SPAWAR  2012)  model  shown  below  in  Figure  21  is  being  applied  to  align  the  systems 
being  developed  and  procured  to  have  traceability  from  the  joint  level  downwards.  Thus 
the  mission  capabilities  are  addressed  at  the  appropriate  level  along  with  the  notional 
evolution  of  capabilities  moving  forward.  For  example  if  a  joint  STK  mission  capability 
is  needed  the  high  level  capabilities  are  defined  in  the  Joint  Capabilities  Development 
System.  The  capabilities  are  broken  down  into  SOS  or  family  of  system  (EOS)  portfolios 
of  capabilities  such  as  communications.  The  SOS  or  EOS  capabilities  are  broken  down 
further  to  the  platform  level.  Finally  the  systems  needed  onboard  the  platform  are  defined 
at  the  lowest  level. 


FoS/SoS 


Joint 

Capabilities 

Development 

System 


t-  ■  t-  - 


Force  CapabWI^  Documents 


Capability 

Evolution 

Planning 


Portfolio 


Capability 

Engineering 


FoS/Sos 

Portfolio 


*  Translates  needed  capability  into  preferred 
solution  set  of  units  and  systems 

*  Supports  AoAs  in  initial  performance  and 
functional  allocation  from  Systems  of  Systems 
context 

*  Identifies  Time-Phased  capability  evolution 
path  and  capability  fielding  plan 


*  Refines  allocations  and  identifies  critical 
interfaces  between  portfolio  systems 

*  Develops  system  performance  document  as 
basis  forflow-down  of  requirements  to 
portfolio  system  specifications 


Portfolio 

Execution 


Portfolio 

Systems 


*  Assesses  progress  of  portfolio 
programs  during  execution 

*  Assesses  the  consequences  of 
Investment  decisions  on  capability 
fielding  plan 


-Time- 


Fielded  Capability 


Capability  Increment  1 


Capability  Increment  2 


Figure  21 .  Naval  Capability  Evolution  Process  Model  (from  SPAWAR  2012,  6) 
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SPAWAR  Systems  Centers  Atlantic  and  Pacific  have  various  resources  they 
develop  and  post  within  their  technical  libraries  available  to  their  workforce.  The 
SPAWAR  SOSE  and  integration  “Vee”  shown  in  Figure  22  is  one  example  of  the 
products  used  to  illustrate  SOSE  processes.  The  Vee  shows  the  division  between  the 
SOSE  and  the  SE  activities.  Starting  from  the  left  the  SOS  activities  follow  a  similar 
process  as  the  NCEP  model  to  break  the  mission  SOS  into  the  platform  SOS  to  the 
constituent  systems.  The  right  side  is  the  validation  and  verification  as  each  system  is 
tested  with  a  platform  SOS  and  ultimately  the  mission  SOS  environment.  The  Systems 
Engineering  Guide  for  Systems  of  Systems  (Director,  Systems  and  Software  Engineering 
2008)  describes  seven  core  elements  to  be  applied  to  the  SOSE  process.  These  are  similar 
to  the  NCEP  model  to  translate  mission  SOS  capabilities  into  specific  requirements  at  the 
SOS  and  systems  levels  (SPAWAR  2012).  The  evolution  of  SOS  is  considered  since 
capabilities  change  over  time.  Continually  assessing  the  capabilities  and  evolving  the 
architecture  become  additional  considerations  where  SOS  engineer  engages  with  the  SOS 
and  systems  teams.  By  applying  the  SOSE  Vee  processes,  changes  can  be  managed  and 
fielded  with  minimal  impact  to  the  larger  SOS. 


Figure  22.  System  of  Systems  Engineering  and  Integration  “Vee”  (from 

Vaneman  and  Budka  2013) 
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Submarine  communications  have  evolved  from  simple  single  function  systems  to 
multi-function  integrated  systems.  The  creation  of  SCSS  proved  a  system  of  systems 
approach  would  work  and  set  the  groundwork  for  establishing  CSRR  as  a  formal  system 
of  systems  program.  As  a  formal  program,  CSRR  has  a  body  of  required  acquisition, 
engineering,  and  logistics  documentation  which  can  be  used  to  support  creating  a  case 
study.  This  literature  review  identified  a  body  of  information  available  concerning  the 
purpose  of  a  case  study  and  the  benefits.  Additionally  there  is  a  significant  amount  of 
information  available  about  systems  and  systems  engineering.  There  is  also  substantial 
evidence  DOD  has  been  moving  toward  system  of  systems  and  system  of  systems 
engineering  as  disparate  capabilities  are  increasingly  integrated.  Despite  this  move,  most 
of  the  literature  regarding  system  of  systems  engineering,  integration  and  policy  has  been 
relatively  recent. 
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III.  METHODOLOGY 


The  methodology  used  for  this  researeh  looked  at  a  logieal  manner  of  providing 
the  neeessary  background  and  context  in  order  to  understand  the  reasons  for  some  of  the 
activities  which  occurred  and  the  overall  impact,  either  positive  or  negative.  The  research 
questions  were  bounded  by  the  purpose  to  aid  in  providing  background  and  context.  The 
purpose  of  this  case  study  was  used  as  a  basis  for  the  research  questions  and  they  are 
listed  below. 

1 .  The  history  of  submarine  communications  leading  up  to  the  creation  of  the 
CSRR  program. 

2.  The  organizational  structure  of  the  CSRR  program. 

3.  The  relationship  with  other  programs  of  record  and  stakeholders. 

4.  System  of  systems  architecture  management. 

5.  The  advantages  and  disadvantages  of  the  CSRR  SOS  approach  within  the 
various  disciplines  (e.g.,  modernization,  integrated  logistics  support  (ITS), 
training,  sustainment,  and  information  assurance  (lA)). 

6.  Process  improvement  initiatives  and  their  impact  in  regard  to  cost, 
schedule  and  performance. 

7.  CSRR’s  ability  to  meet  future  mission  requirements  while  supporting 
current  missions 

The  wide  scope  for  this  case  study  is  necessary  given  the  history  of  submarine 
communications  leading  up  to  CSRR  and  amount  of  activity  that  has  occurred. 
Maintaining  focus  on  the  purpose  ensures  we  are  addressing  the  correct  topics  while  the 
scope  ensures  the  questions  are  reasonably  bounded  so  they  could  be  answered  in  a 
reasonable  manner.  The  scope  was  applied  in  the  methodology  and  is  listed  below. 

1 .  The  development  of  submarine  communications  from  its  initial  beginnings 
up  through  the  deployment  of  CSRR  increment  one  version  three. 

2.  The  organizational  structure  of  the  CSRR  program  to  include  the  design 
and  development  group,  production  and  installation  group,  ITS  and 
training  groups,  lA  groups,  and  sustainment  group. 

3.  The  version  development  process,  its  strengths  and  weaknesses. 

4.  SOS  architecture  management  with  other  programs  of  record  and  portfolio 
capability  management,  the  relationships  with  the  other  programs  of 
record  and  the  warfighter. 
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5.  The  advantages  and  disadvantages  of  the  CSRR  system  of  systems 
approach  regarding  ILS,  training,  production,  installation  (synchronization 
of  installations  into  block  upgrades),  lA  and  sustainment. 

6.  Assessment  of  requirements  in  a  changing  environment  with  regard  to  the 
Undersea  Connectivity  Roadmap,  Design  for  undersea  warfare,  PEO  C4I 
Master  Plan,  and  the  way  ahead  for  considering  disruptive  technologies. 

7.  An  evaluation  of  the  process  improvement  initiatives  and  their  influence 
and  impact  in  regard  to  cost,  schedule  and  performance. 

8.  The  future  of  CSRR  in  today’s  environment  and  tomorrow  up  through 
2030. 

The  methodology  for  this  case  study  consisted  of  the  following  activities. 

1 .  Investigation  into  other  case  studies  to  determine  if  other  researchers  had 
performed  similar  work  and  confirm  if  a  case  study  would  be  an 
appropriate  approach.  Review  of  other  case  studies  did  indicate  similar 
work  had  been  done  and  several  case  studies  addressed  SOS  issues. 

2.  Investigation  into  Navy  and  specifically  PEO  C4I  archives  to  determine  if 
any  case  studies  had  been  written.  No  case  studies  could  be  located  in  the 
SPA  WAR  and  PEO  archives  or  technical  libraries. 

3.  Review  of  the  DOD  acquisition  and  program  documentation  regarding 
SOS,  defense  acquisition  requirements,  systems  and  system  of  systems 
principles  and  how  to  measure  the  characteristics  of  a  SOS.  A  large 
amount  of  information  is  available  for  acquisition,  systems  engineering 
and  systems  engineering  principles  both  locally  and  online.  Most  of  the 
system  of  systems  literature  has  been  developed  in  the  last  14  years. 

4.  Perform  an  in  depth  analysis  of  the  CSRR  program  documentation.  This 
includes  the  formal  program  documentation  and  minutes  from  the  various 
integrated  product  teams  (IPT)  supporting  the  program.  The 
documentation  provided  insight  to  the  history,  requirements  and  policies 
for  managing  the  CSRR  program. 

5.  Conduct  selected  interviews  with  subject  matter  experts  (SME)  with 
regard  to  developing  and  managing  a  SOS  program  and  the  individual 
systems  supporting  the  SOS.  Interviews  were  conducted  with  key 
members  of  the  engineering  and  production  teams. 

6.  Synthesize  the  information  to  capture  lessons  learned  (or  learning 
principles),  develop  conclusions  and  make  recommendations  for  further 
consideration.  A  derivative  of  the  Eriedman-Sage  framework  will  be  used 
since  contractor  involvement  is  limited. 
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Initial  research  began  identifying  background  information  concerning  case  studies 
related  to  systems  engineering  and  system  of  systems  engineering  to  determine  if  a  case 
study  approach  would  be  appropriate.  Eleven  case  studies  were  identified  and  read.  These 
case  studies  are  listed  in  Table  10. 

These  case  studies  were  reviewed  in  order  to  understand  the  reasons  they  were 
performed,  how  they  were  accomplished  and  what  significant  lessons  were  identified. 
Research  of  these  case  studies  looked  for  common  themes.  Several  case  studies 
acknowledged  they  were  part  of  a  SOS  (GPS,  TBMCS).  The  learning  principles  were 
compared  against  the  lessons  learned  from  the  CSRR  program. 

Most  of  the  case  studies  used  the  Friedman  and  Sage  framework  to  capture 
learning  principles.  Several  of  these  case  studies  included  their  learning  principals  as  part 
of  the  whole  document  and  the  remainder  provided  in  a  separate  executive  summary. 
Examples  of  learning  principles  identified  were  compared  against  the  CSRR  program 
lessons  learned  to  determine  if  commonalities  existed. 


Table  10.  Case  Studies  Used  in  This  Research 


Case  Study 

Sponsor 

KC-135  Flight  Simulator 

Air  Force  Center  for  Systems  Engineering 

Global  Positioning  System 

International  Space  Station 

F-1 1 1  Aardvark 

C-5A  Galaxy 

Hubble  Space  Telescope 

A- 10  Thunderbolt  II 

Theater  Battle  Management  Core  System 

Global  Hawk 

Miniature  Seeker  Technology  Integration 

Virginia  Polytechnic  Institute  and  State 
University 

V-22 

U.S  Army  Command  and  General  Staff 
College 
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Online  research  of  the  PEO  and  SPAWAR  process  activity  libraries,  Naval 
Systems  Engineering  Resource  Center  and  Systems  Engineering  Environment,  located 
applicable  SE  and  SOSE  policy  documents.  However,  no  case  studies  were  located  in  the 
repositories.  While  case  studies  are  available  through  other  online  means  there  is  no 
central,  easily  accessible  source  for  SPAWAR  and  PEO  C4I  engineering  and  program 
personnel. 

Several  SMEs  were  interviewed  to  capture  their  insights  regarding  the  design, 
production,  and  sustainment  of  CSRR  as  a  SOS.  NUWC  and  SSC  LANE  maintain  the 
core  teams  of  SMEs  responsible  for  the  design,  development,  testing,  production  and 
sustainment.  The  sample  size  of  this  research  was  limited  to  the  people  involved  with  the 
CSRR  program  for  the  last  ten  years.  The  CSRR  chief  engineer,  technical  project 
manager,  design  agent  and  production  agent  were  interviewed  to  capture  their 
responsibilities  and  insight  to  design  and  manage  CSRR  as  a  SOS.  These  interviews  took 
place  via  email  and  telephone.  The  interviews  were  limited  to  the  background  of 
submarine  communications  immediately  preceding  CSRR  and  development  from  VO 
through  V3. 

CSRR  is  a  recognized  acquisition  program  and  maintains  the  required  acquisition, 
technical,  and  logistics  documentation.  The  CSRR  documents  listed  in  Tables  11  through 
13  were  reviewed  to  ascertain  what  information  could  be  used  to  support  a  case  study. 
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Table  1 1 .  CSRR  Acquisition  Documents  Reviewed 


Document 

Purpose  of  Document 

Acquisition/Programmatic 

Mission  needs  statement  for  the  Integrated 
Maritime  Communications  System  (CNO 
N81  1995) 

Original  statement  of  requirement  to  fill  a 
capability  gap.  This  would  be  the  Initial 
Capabilities  Document  under  the  JCIDS 
process. 

Common  Submarine  Radio  Room 
Capabilities  Production  Document 
(PMW770  2006) 

Defines  the  production  elements 
applicable  to  CSRR  increment  one 

Submarine  Exterior  Communications 

System  Capstone  Requirements  Document 
(CNO  N8  1998;  CNO  N8  2003) 

Similar  to  a  Capability  Description 
Document  (CDD)  it  defines  the  key 
performance  parameters  and  key  systems 
attributes 

Submarine  Communications  Master  Plan 
(CNON87  1995;  CNON61  andN87  1998) 

Part  of  the  submarine  force  strategic  plan 
consolidating  current  and  future 
communications  requirements  in  support 
of  the  budget  planning  process 

CSRR  Acquisition  Strategy/ Acquisition 

Plan  (PMW770  2008) 

Outlines  the  business  and  technical 
acquisition  management  approach  in  order 
to  achieve  the  program  objectives  within 
the  allotted  resources 

Concept  of  operations  for  CSRR  Incl  V2 
and  Inc  I  V3  (PMW770  201  Ib) 

Defines  the  overall  intent  of  a  particular 
mission  or  strategy.  The  CONOPS 
provides  a  high  level  view  of  how  a 
system  will  be  employed 

Design  for  Undersea  Warfare  (Richardson, 
Caldwell  and  Breckenridge  2011) 

Commander  Submarine  Eorce  guidance 
for  executing  their  “lines  of  effort”  for 
maintaining  readiness  for  operations, 
maximizing  effectiveness  during 
deployments  and  develop  future 
capabilities 

PEO  C4I  Strategic  Plan  2013-2018 
(Burroughs  2012) 

PEO  guidance  for  delivering  capability 
and  reducing  variance 

Undersea  Connectivity  Roadmap 
(Hendricks  and  Duffy  2012) 

Undersea  enterprise  vision  of  future 
capabilities  for  communications  and 
undersea  connectivity 

DOD  Inspector  General  (IG)  report  on 

CSRR  (2005) 

IG  investigation  into  the  performance  of 
the  CSRR  program 
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Table  12.  CSRR  Engineering  Documentation  Reviewed 


Technical 

System  Engineering  Plan  (PMW770  2007) 

Defines  the  programs  strategy  for 
managing  all  of  the  technical  activities 
related  to  systems  engineering 

Test  and  Evaluation  Master  Plan  (PMW770 
2012a) 

Defines  the  overall  test  strategy  and 
resources  needed  to  accomplish  all 
required  developmental  and  operational 
testing 

System  Subsystem  Design  Description  for 
CSRR  Incl  V3  SSBN  (NUWC  2008),  VA 
(NUWC  2012)  and  LA  (NUWC  2011) 

Provides  program  background  and 
describes  the  physical  and  functional 
designs  of  the  capabilities  within  each 
version. 

Systems  Design  Verification  Test  Plan  for 
CSRR  Incl  V3  LA  (NUWC  2011),  VA 
(NUWC  2012b),  SSGN  (PMW770  2013a), 
and  SSBN  (PMW770  2013b) 

Defines  the  testing  parameters  to  verify 
CSRR  can  meet  all  performance 
requirements  outlined  in  the  CPD. 

Supports  the  TEMP  as  part  of  the  overall 
testing  strategy 

CSRR  System  Requirements,  Design, 
Integration  and  Testing  Process  (Ross  2013) 

An  internal  document  developed  to  aid 
new  team  members  in  familiarizing 
themselves  with  the  CSRR  program 

CSRR  Circuit  Matrix  (PMW770  2014) 

A  formal  agreement  between  PMW770 
and  the  submarine  force  which  details 
which  circuits  will  be  supported  by  a 
particular  CSRR  version  and  platform 
class 

Table  13.  Logistics  and  Training  Documentation  Reviewed 


Logistics 

Integrated  Logistics  Support  Plan 
(PMW770  2008b) 

Supports  the  CSRR  acquisition  strategy 
and  defines  the  sustainment  strategy  for 
the  overall  program 

CSRR  Navy  Training  Systems  Plan  (CNO 
N2N6  2012) 

Service  specific  training  systems  plan 
which  serves  as  the  agreement  between  the 
program  stakeholders 

CSRR  Reliability,  Maintainability,  and 
Availability  (RMA)  reports 

Periodic  reports  from  the  Navy’s 
independent  assessor  for  RMA  data 
collected  from  maintenance,  material  and 
management  (3M)  system 
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In  2008  the  Deputy  Seeretary  of  Defense  issued  a  policy  directing  all  DOD 
activities  to  begin  using  Lean  Six  Sigma  (LSS).  The  CSRR  program  performed  a  number 
of  improvement  activities  in  order  to  improve  meeting  cost,  schedule  and  performance. 
These  improvement  activities  were  reviewed  as  well  to  determine  what  issues  occurred, 
their  root  cause  and  their  solutions.  Table  14  is  a  summary  of  the  continuous 
improvement  events  using  LSS. 

In  order  to  gain  a  better  understanding  of  the  SE  and  SOSE  processes  used  by 
other  DOD  and  government  organizations  an  investigation  into  the  differences  and 
benefits  between  systems  engineering  and  systems  of  systems  engineering  was 
accomplished.  These  references  were  located  on  the  INCOSE  website.  Defense 
Acquisition  Portal  Acquisition  Community  Connection,  Dudley  Knox  library,  and  other 
recognized  sources  such  as  MITRE  and  Massachusetts  Institute  of  Technology  (MIT). 
Table  15  is  a  summary  of  the  types  of  documentation  examined. 


Table  14.  Continuous  Process  Improvement  Activity  Documentation  Reviewed 


Process  Improvement  Events 

CSRR  version  value 
stream  analysis  (VS A) 

Eean  event  to  evaluate  how  to  shorten  the  CSRR 
development  cycle  following  the  deployment  of  CSRR  VO 

REDACS  reliability 
assessment 

Eean  six  sigma  (ESS)  event  to  analyze  system  failures, 
identify  the  root  cause  and  develop  improvements  for 
increasing  system  Aq 

CSRR  testing 
streamlining  rapid 
improvement  event 

ESS  event  to  identify  non-value  added  activities  in  the  test 
and  certification  process  and  identify  areas  where  testing  can 
be  leveraged 

CSRR  modernization 
other  productivity 
improvement  event 

ESS  event  to  capture  the  benefits  of  performing  consolidated 
installations  during  available  modernization  periods 

CSRR  integration 
streamlining  rapid 
improvement  event 

ESS  event  to  revisit  the  original  CSRR  VSA  and  attempt  to 
address  configuration  variance  issues  identified  during  the 
employment  of  CSRR  V3 
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Table  15.  Other  Applicable  Engineering  Documents  Reviewed 


Other  engineering  documents 

INCOSE  Systems  Engineering  Book  of  Knowledge  (Adcock  2014) 

OSD  Systems  Engineering  Guide  for  System  of  Systems  (Director,  Systems  and 
Software  Engineering  2008) 

DSMC  Systems  Engineering  Eundamentals  Guide  (DSMC  2001) 

NASA  Systems  Engineering  Guide  (NASA  2007) 

MITRE  System  Engineering  Guide  (MITRE  2014) 

DOD  Systems  Engineering  annual  reports  released  by  the  GAO 
SOS  modeling  and  acquisition 


The  methodology  outlined  the  plan  of  action  to  identify  and  accumulate 
information  to  complete  the  case  study.  Once  all  of  the  information  was  identified  and 
collected,  the  initial  analysis  developed  the  framework  for  the  case  study.  Research  using 
the  information  from  the  systems  engineering  classes  provided  a  starting  point  for 
developing  the  research  questions  and  a  plan  to  answer  them.  Further  investigation  of  the 
local  and  online  resources  including  SPAWAR,  PEO  C4I,  PMW770,  Dudley  Knox 
library  and  the  Defense  Technical  Information  Center  portal  located  CSRR  program 
documentation  and  related  SE  and  SOS  information.  The  information  was  collected  and 
analyzed  to  answer  several  of  the  questions  presented  in  this  thesis  about  defining  CSRR 
as  a  system  of  systems.  The  results  in  turn  derived  further  answers  to  the  remaining 
questions  for  effectively  developing  and  managing  comparable  system  of  systems. 
Lessons  learned  or  learning  principles  were  validated  and  any  new  ones  were  identified. 
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IV.  APPLICATION  OF  METHODOLOGY 


The  methodology  outlined  in  the  previous  ehapter  will  answer  the  researeh 
questions.  The  results  were  used  to  identify  learning  principles  applicable  to  CSRR  as  a 
system  of  systems.  These  learning  principles  were  also  assessed  to  determine  if  they 
could  be  extended  to  other  programs  or  system  of  systems.  A  number  of  these  can  serve 
as  an  opportunity  to  avoid  mistakes  and  identify  prospects  that  would  otherwise  be 
missed.  Each  of  the  research  questions  will  be  answered  in  more  detail. 

A,  EVALUATION  OF  THE  CASE  STUDIES 

The  first  step  to  developing  this  case  study  began  by  reading  other  case  studies  to 
determine  what  they  are  and  decide  if  using  a  case  study  approach  would  be  appropriate. 
Friedman  and  Sage  (2003,  84-86)  stated  case  studies  serve  a  valuable  purpose  by 
exposing  a  student  to  real  world  examples  of  systems  engineering.  Their  stated  position  is 
a  case  study  is  an  “empirical  inquiry  that  investigates  contemporary  phenomenon  within  a 
real-life  context,  especially  when  boundaries  between  the  phenomenon  and  context  are 
not  clear  and  multiple  sources  of  evidence  is  used”  (Friedman  and  Sage  2003,  85). 

There  are  several  benefits  of  performing  a  case  study  (Friedman  and  Sage  2003, 
85).  Understanding  the  how  and  why,  revealing  the  detailed  information  surrounding  an 
event,  and  allows  exploration  of  a  topic  when  a  strong  theory  may  not  be  available.  The 
sources  of  evidence  available  to  conduct  research  include  looking  at  documentation, 
records,  interviews  and  observations.  Using  more  than  one  approach  can  add  detail  and 
context.  One  of  several  approaches  such  as  pattern  matching,  explanation  building,  logic 
models,  and  time  series  or  cross  case  analysis  can  be  used  to  analyze  the  collected 
information. 

An  initial  search  was  performed  to  determine  how  much  information  about  case 
studies  existed.  Using  Google,  a  search  for  “case  studies”  returned  over  25  million 
possible  results.  Extending  this  further  to  include  the  term  “system  of  systems”  narrowed 
the  results  to  over  350,000.  The  search  for  identified  case  studies  written  by  the  Air  Force 
and  NASA  concerning  the  Hubble  space  telescope,  F-1 1 1,  Global  Hawk,  TBMCS,  MTSI 
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and  others.  A  search  of  the  Dudley  Knox  Library  identified  over  9,000  possible  results. 
For  the  term  “navy  system  engineering  case  studies”  identified  96  possible  candidates. 
Performing  a  more  detailed  search  for  a  “navy  engineering  system  of  systems  case 
studies”  identified  only  five  hits,  none  of  which  pertained  to  the  search.  Searches  of  the 
Defense  Acquisition  University  portal  identified  a  number  of  case  studies  including  the 
ones  used  in  this  study. 

The  SPAWAR  chief  engineers  web  portal  maintains  the  process  activity  library  as 
a  repository  and  workspace  for  SE  and  SOSE  documentation  related  to  the  information 
dominance  enterprise  architecture  (IDEA)  (SPAWAR  2014a).  The  IDEA  is  attempting  to 
define  the  way  forward  for  developing  and  delivering  future  C4I  strategic  level  objectives 
in  support  of  information  dominance  capability  to  the  warfighter.  Eigure  23  shows  how 
the  IDEA  disassembles  the  enterprise  SOS  architecture  into  more  discrete  requirements 
and  systems.  Using  a  process  similar  to  the  NCEP  and  the  Vee  the  objective  requirements 
are  decomposed  to  the  mission  and  service  requirements.  These  requirements  are  then 
assigned  to  the  specific  programs  to  develop  and  acquire  the  applicable  systems. 
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Eigure  23.  SPAWAR  Information  Dominance  Enterprise  Architecture 
Requirements  Tree  (from  SPAWAR  2014b) 
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Additional  searches  of  the  PEO  C4I  and  SPAWAR  technical  repositories  revealed 
no  SPAWAR  systems  case  studies  were  available.  SSC  PAC  maintains  a  technical  library 
to  support  their  science  and  engineering  activities,  but  no  case  studies  were  available  for 
systems  under  their  purview.  SSC  LANT  maintains  a  limited  technical  library  primarily 
devoted  to  holding  various  drawings.  The  CSRR  design  agent  was  also  asked  if  NUWC 
Newport  maintains  a  technical  library.  Their  technical  library  mainly  handles  drawings 
and  technical  publications  requested  by  the  staff  (Steve  Devin  2014,  email  questionnaire). 
The  availability  of  any  related  engineering  case  studies  could  not  be  confirmed. 

The  case  studies  confirmed  the  approach  would  be  appropriate  for  the  goals  of 
this  study.  Ten  case  studies  used  a  Friedman  and  Sage  framework  or  a  slight  derivative 
while  the  MSTI  and  V-22  created  a  different  approach  to  lessons  learned.  Several  of  the 
case  study  learning  principles  were  provided  in  an  executive  summary  and  were  not 
available.  For  these  case  studies,  such  as  the  KC-135,  significant  items  were  identified 
and  presented  as  learning  principles.  In  all  cases  there  were  other  topics  which 
represented  possible  learning  principles  as  well.  For  this  study  the  evaluation  was  limited 
to  the  ones  presented  in  the  executive  summary  or  extracted  if  a  summary  was  not 
available.  An  initial  evaluation  identified  the  framework  domains  addressed  from  each 
case  study  as  seen  in  Table  16. 


Table  16.  Summary  of  Domain  Areas  Impacted  by  Each  Case  Study 


Programs 


Framework  Domain  Areas 

A-10 

TBMCS 

B-2 

C-5A 

Hubble 

ISS 

GPS 

F-lIl 

GH 

MTSI 

KC-135 

A.  Requirements  Definition  and 
Management 

X 

X 

X 

XX 

X 

X 

X 

XX 

X 

B.  Systems  Architecting  and 
Conceptual  Design 

XX 

X 

X 

XX 

X 

X 

X 

X 

X 

C.  DetaUed  System  and  Subsystem 
Design  and  Implementation 

X 

X 

X 

X 

X 

XX 

X 

X 

X 

D.  Systems  and  Interface  Integration 

X 

X 

X 

X 

XX 

X 

X 

E.  Validation  and  Verification 

X 

X 

X 

F.  System  Deployment  and  Post 
Deployment 

X 

G.  Life  Cycle  Support 

X 

X 

X 

H.  Risk  Management 

X 

X 

X 

X 

X 

X 

X 

I.  System  and  Program  Management 

X 

XX 

X 

XXX 

XX 

XX 

X 

XXX 

X 

77 


All  of  the  case  studies  used  the  Friedman  and  Sage  framework  except  for  the 
MSTI  and  the  V-22.  The  MSTI  lessons  were  applied  to  the  Friedman  and  Sage  domains 
based  on  this  researcher’s  assessment.  The  V-22  case  study  was  not  written  to  assess  the 
application  of  systems  engineering  principles  so  it  was  removed  from  further 
consideration.  The  eleven  case  studies  were  then  assessed  to  determine  if  there  were 
correlations  between  the  earlier  case  studies  and  CSRR.  Each  case  study  identified  areas 
which  were  related  to  similar  issues  with  CSRR.  For  example  the  TBMCS  team  had  not 
engaged  with  the  user  community  to  develop  a  CONOPS,  the  systems  architecture  did 
not  have  adequate  detail  and  interface  management  was  poor  or  nonexistent.  Another 
example  using  the  B-2  identified  a  failure  to  recognize  a  maturity  issue  needed  attention, 
which  resulted  in  a  five  month  delay  and  missing  the  Critical  Design  Review  (CDR) 
milestone  (Griffin  and  Kinnu  2007,  53).  The  F-111  described  the  problems  faced  by  the 
engineering  team  when  a  major  stakeholder  chose  to  pull  out  of  the  effort  which  had 
lasting  and  substantial  impacts  on  the  program  (Richey  2005,  30-31). 

Similar  challenges  exist  for  CSRR  in  that  individual  systems  may  not  have  a 
CONOPS,  or  if  they  do,  it  is  not  uncommon  for  them  to  conflict  with  another.  This  might 
appear  to  be  in  conflict  with  the  JCIDS  but  a  large  number  of  programs  had  been 
developed  prior  to  JCIDS  being  fully  implemented.  If  the  program  was  post  milestone 
they  were  grandfathered  and  the  existing  documentation  was  accepted  to  support  that 
particular  milestone  decision.  Oversight  of  ACAT  I  programs,  minimal  oversight  of  other 
ACAT  programs  beyond  budgetary  cycle  requirements  and  infrequent  reviews  of  systems 
once  in  sustainment  deter  assessing  if  a  system  is  aligned  to  a  mission  SOS. 

Systems  architecture  is  defined  for  the  individual  system  but  illustrates  itself  as 
operating  as  a  piece  in  the  global  architecture  and  not  necessarily  in  the  intermediate 
architecture  which  would  include  CSRR  or  SWFTS.  Interfaces  are  usually  defined  at  the 
system  level  but  routinely  do  not  interoperate  with  other  systems,  forcing  some  type  of 
middleware  solution.  Several  authors  acknowledged  they  were  working  within  a  SOS 
architecture  while  conducting  their  case  study  on  the  area  of  interest.  The  specific 
learning  principles  or  lessons  learned  from  these  case  studies  are  listed  below  in  Table 
17. 
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Table  17.  Learning  Principles  by  Case  Study  and  the  Associated  Domains 

Affected 


System  Case  Study 

Associated  Friedman  and  Sage 
Domains 

A- 10  Thunderbolt  II  Warthog  (Jacques  2010) 

LP-I  The  system  concept  and  preliminary  design 
must  follow,  not  precede,  the  mission  analysis. 

A.  Requirements  Definition 
and  Management 

B.  Systems  Architecture  and 
Conceptual  Design 

C.  System  and  Subsystem 
detailed  design  and 
implementation 

LP-2;  Prototyping  can  be  used  to  help  manage 
technical  and  cost  risk  at  the  system,  subsystem,  and 
component  level. 

H.  Risk  Assessment  and 
Management 

LP-3:  Clear  lines  of  responsibility  must  be 
established  to  ensure  successful  integration, 
especially  when  multiple  programs  are  involved. 

D.  Systems  Integration  and 
Interface 

LP-4;  The  government  must  ensure  the  contractor  is 
able  to  “Walk  the  Talk”  when  it  comes  to  production. 

G.  Life  Cycle  Support 

H.  Risk  Assessment  and 
Management 

LP-5;  Successful  design,  development  and  production 
is  not  enough  to  sustain  a  system  throughout  its  life 
cycle. 

F.  Deployment  and  Post 
Deployment 

LP-6;  If  the  politics  do  not  fly,  the  system  never  will. 

B.  Systems  Architecture  and 
Conceptual  Design 

Theater  Battle  Management  Control  System  (Collens  and  Krause  2005) 


LP-1;  The  government  did  not  produce  a  Concept  of 
Operations,  key  operational  performance  parameters, 
or  a  system  specification  for  the  contractor.  The 
requirements  baseline  was  volatile  up  to  system 
acceptance,  which  took  place  after  operational  test 
and  evaluation. 

A.  Requirements  Definition 
and  Management. 

LP-2;  The  system  architecture  was  defined  at  too 
high  a  level,  which  had  a  tremendous  impact  on 
system  design  and  development. 

B.  System  Architecture. 

LP-3:  The  system  and  subsystem  design  was  severely 
hampered  by  the  complexity  of  legacy  applications, 
misunderstanding  of  the  maturity  and  complexity  of 
commercial  and  third  party  software  products,  and  a 
lack  of  understanding  of  how  the  system  would  be 
employed  by  the  user. 

C.  System/Subsystem  Design 

LP-4;  Systems  and  interface  integration  was  highly 
complex.  The  external  system  interfaces  were  not 
managed  and  were  often  impossible  to  test  at  the 

D.  Systems  Interface  and 
Integration 
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System  Case  Study 

Associated  Friedman  and  Sage 
Domains 

contractor’s  facility. 

LP-5;  The  lack  of  a  firm  requirements  baseline  made 
validation  and  verification  very  difficult.  Not  being 
able  to  replicate  the  operational  environment  prior  to 
acceptance  test  created  severe  problems. 

E.  Validation  and  Verification 

B-2  Spirit  Bomber  (Griffin  and  Kinnu  2007) 

LP-1:  Integration  of  the  Requirements  and  Design 
Processes.  A  key  aspect  of  the  implementation  of  the 
systems  engineering  process  was  the  integration  of 
the  SPO  requirement’s  team  with  the  contractors’ 
work  breakdown  structure  task  teams  into  a  cohesive 
program  effort. 

A.  Requirements  Definition 
and  Management. 

LP-2:  Work  breakdown  structure  task  teams  and 
functional  hierarchy.  A  well-defined  contract  work 
breakdown  structure  stipulated  the  entire  program 
content  and  tasking. 

1.  System  and  Program 
Management 

LP-3;  Air  vehicle  reconfiguration.  When  the 
identification  of  a  major  aeronautical  control 
inadequacy  was  discovered  just  four  months  prior  to 
formal  configuration  freeze,  an  immediate  refocus  of 
the  task  teams  was  required. 

B.  Systems  Architecting  and 
Conceptual  Design 

LP-4:  Subsystem  maturity.  The  effect  of  the 
reconfiguration  on  the  maturity  of  all  the  air  vehicle 
subsystems  (flight  control,  environmental  control, 
electrical,  landing  gear,  etc.)  was  far  greater  than 
projected. 

C.  System  and  Subsystem 
detailed  design  and 
implementation 

LP-5;  Risk  planning  and  management.  The  program 
was  structured  so  that  risks  affecting  the  viability  of 
the  weapons  system  concept  were  identified  at 
contract  award  and  were  structured  as  part  of  the 
program  and  work  breakdown  structure  work  plans. 

H.  Risk  Assessment  and 
Management 

C-5A  Galaxy  (Griffin  2004) 

LP-1;  The  process  for  developing  and  documenting 
the  system  performance  requirements  involved  the 
user,  planners,  developers,  and  technologists  from 
both  the  government  and  industry  in  a  coordinated  set 
of  trade  studies.  It  resulted  in  a  well-balanced,  well- 
understood  set  of  requirements  that  fundamentally 
remained  unchanged  throughout  the  program. 

A.  Requirements  definition  and 
management 

LP-2;  The  total  package  procurement  concept  (TPPC) 
employed  by  the  government  required  a  fixed-price, 
incentive  fee  contract  for  the  design,  development, 
and  production 

H.  Risk  Assessment  and 
Management 

I.  System  and  Program 
Management 
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System  Case  Study 

Associated  Lriedman  and  Sage 
Domains 

LP-3;  A  weight  empty  guarantee  was  included  in  the 
specification  as  a  performance  requirement  and  in  the 
contract  as  a  cost  penalty  for  overweight  conditions 
of  delivered  aircraft.  The  weight  empty  guarantee 
dominated  the  traditional  aircraft  performance 
requirements  (range,  payload,  etc.),  increased  costs, 
and  resulted  in  a  major  shortfall  in  the  wing  and 
pylon  fatigue  life.  The  stipulation  of  a  weight  empty 
guarantee  as  a  performance  requirement  had  far- 
reaching  and  significantly  deleterious  unintended 
consequences. 

A.  Requirements  Definition 
and  Management 

LP-4;  The  system  program  office  employed 
independent  review  teams  to  assemble  national 
experts  to  examine  the  program  and  provide 
recommendations  to  the  government.  These  problem¬ 
solving  teams  were  convened  to  gamer  the  best 
advice  in  particular  technical  areas:  stmcture  design 
and  technology,  and  designs  to  achieve  useful  service 
life. 

I.  System  and  Program 
Management 

Hubble  Space  Telescope  (Mattice  2003) 

LP-1:  Early  and  full  participation  by  the  customer/ 
user  throughout  the  program  is  essential  to  success. 

A.  Requirements  definition  and 
management 

LP-2:  The  use  of  pre-program  trade  studies  to  broadly 
explore  technical  concepts  and  alternatives  is 
essential  and  provides  for  a  healthy  variety  of  inputs 
from  a  variety  of  contractors  and  government. 

B.  Systems  Architecting  and 
Conceptual  Design 

LP-3:  A  high  degree  of  systems  integration  to 
assemble,  test,  deploy,  and  operate  the  system  is 
essential  to  success  and  must  be  identified  as  a 
fundamental  program  resource  need  as  part  of  the 
program  baseline. 

C.  System  and  Subsystem 
detailed  design  and 
implementation 

D.  Systems  Interface  and 
Integration 

LP-4:  Life  cycle  support  planning  and  execution  must 
be  integral  from  day  one,  including  concept  and 
design  phases.  The  results  will  speak  for  themselves. 

B.  Systems  Architecting  and 
Conceptual  Design 

G.  Life  Cycle  Support 

LP-5:  Lor  complex  programs,  the  number  of  players 
(government  and  contractor)  demands  that  the 
program  be  stmctured  to  cope  with  high  risk  factors 
in  many  management  and  technical  areas 
simultaneously. 

I.  System  and  Program 
Management 

International  Space  Station  (Stockman,  Boyle  and  Bacon  2010) 


LP-1:  Systems  engineering  involves  communications, 
critical  to  international  partnerships,  so  before 
worrying  about  technical  interfaces,  make  sure  the 

I.  System  and  Program 
Management 
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System  Case  Study 

Associated  Friedman  and  Sage 
Domains 

integrated  product  teams  and  communication 
bandwidth  between  partners  are  optimal. 

LP-2;  Maintaining  a  high  level  of  competent  and 
experienced  personnel  over  a  two  decade  long 
program  requires  strategic  level  planning  and 
execution  of  workforce  planning. 

I.  System  and  Program 
Management 

LP-3:  Do  not  be  so  ready  to  chase  revolutionary 
designs  over  evolutionary  designs.  A  key  lesson  from 
Russian  experience  (such  as  the  Soyuz)  is  that  it  is 
often  less  risky  to  stay  with  a  known  design  and 
provide  minor  improvements. 

B.  System  Architecture  and 
Conceptual  Design 

LP-4;  Multi-element  integrated  testing  with  actual 
hardware,  high  fidelity  simulators  and  connectors  is 
critical  and  must  be  in  the  program  from  day  one 

E.  Validation  and  Verification 

LP-5;  In  an  ISS  like  project  where  so  many  different 
countries  and  companies  contribute  hardware  and 
software,  the  interfaces  must  be  extremely  simple. 

C.  System  and  Subsystem 
detailed  design  and 
implementation 

D.  Systems  Interface  and 
Integration 

LP-6:  Do  not  be  too  quick  to  allow  partners  (or 

NASA)  to  start  building  modules  or  expensive 
experiments  too  far  in  advance  of  locking  in  schedule 
and  program  baseline 

A.  Requirements  Definition 
and  Management 

I.  System  and  Program 
Management 

Global  Positioning  System  (Griffin  and  O’Brien  2007) 


LP-I;  Programs  must  strive  to  staff  key  positions 
with  domain  experts. 

I.  System  and  Program 
Management 

LP-2;  The  systems  integrator  must  rigorously 
maintain  program  baselines. 

C.  System  and  Subsystem 
detailed  design  and 
implementation 

D.  Systems  Interface  and 
Integration 

LP-3:  Achieving  consistent  and  continuous  high-level 
support  and  advocacy  helps  funding  stability,  which 
impacts  systems  engineering  stability. 

I.  System  and  Program 
Management 

LP-4;  Disciplined  and  appropriate  risk  management 
must  be  applied  throughout  the  life  cycle. 

H.  Risk 

Assessment/Management 

LP-5;  The  GPS  case  study  highlights  the  need  for 
systems  thinking  throughout. 

B.  System  Architecture  and 
Conceptual  Design 

C.  System  and  Subsystem 
detailed  design  and 
implementation 

D.  Systems  Interface  and 
Integration 
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Associated  Friedman  and  Sage 
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F-111  (Richey  2005) 

LP-1;  Ill  conceived,  difficult  to  achieve  requirements 
and  attendant  specifications  made  the  system 
development  extremely  costly,  risky  and  difficult  to 
manage.  The  state  of  technical  maturity  was  not  well 
understood  by  either  contractor  or  government  in  the 
case  of  inlet-engine  compatibility  (dynamic 
distortion)  and  structural  fracture  mechanics  of  brittle 
materials. 

A.  Requirements  Definition 
and  Management 

LP-2:  Systems  engineering  managers  were  not 
allowed  to  make  important  tradeoffs  that  needed  to  be 
made  in  order  to  achieve  an  effective  design  that  was 
balanced  for  performance,  cost  and  mission 
effectiveness  and  the  attendance  risk  and  schedule 
impacts.  The  government  provided  the  systems 
architecture  specifications  and  the  contractor 
responded,  although  there  were  concerns  expressed 
by  Navy  and  Air  Force  analysts  that  the  disparate 
range  of  system  architecture  requirements  could  be 
met  while  maintaining  the  required  level  of 
commonality. 

B.  System  Architecture  and 
Conceptual  Design 

D.  Systems  Interface  and 
Integration 

LP-3;  The  program  suffered  from  poor 
communications  between  Air  Force  and  Navy 
technical  staffs  and  from  over  management  by  the 
Secretary  of  Defense  and  The  Director,  Defense 
Research  and  Engineering  and  received  intense 
congressional  scrutiny,  restricting  the  program  office 
from  applying  sound  systems  engineering  principles. 

I.  System  and  Program 
Management 

LP-4;  The  F-111  had  areas  of  risk  or  deficiency  that 
came  to  light  during  research,  design,  testing  and 
evaluation  even  though  there  was  a  low  perceived 
risk  in  the  design.  The  development  program 
introduced  concurrency  between  design  validation 
and  verification  and  production  to  accelerate  the 
program  schedule. 

H.  Risk 

Assessment/Management 

LP-5:  Cancellation  of  the  Navy  version  after  the  joint 
design  was  frozen  and  production  of  the  Air  Force 
version  was  in  progress  had  a  lasting  impact  on  the 
F-111  performance  and  cost. 

I.  System  and  Program 
Management 

Global  Hawk  (Kinzig  2010) 

LP-1;  The  Joint  Program  Office  was  a  very  small, 
austere  organization,  purposely  sized  that  way  to 
ensure  minimal  oversight  by  the  Government  and 

I.  System  and  Program 
Management 
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System  Case  Study 

Associated  Friedman  and  Sage 
Domains 

provide  a  significant  degree  of  autonomy  to  the 
contractors. 

LP-2;  The  program  developed  a  set  of  desired 
performance  characteristics  that  were  defined  in 
terms  of  a  range  of  values  considered  acceptable.  The 
parameters  were  labeled  as  goals,  either  as  Primary 
Objective,  Objective,  or  Desired.  This  approach  gave 
the  contractor  the  latitude  and  responsibility  to  define 
the  balance  among  the  desired  performance 
parameters,  so  the  user  would  receive  the  “biggest 
bang  for  the  buck.”  This  freed  the  Joint  Program 

Office  from  closely  tracking  the  contractor’s  progress 
in  meeting  a  large  number  of  individual  performance 
specifications.  The  Joint  Program  Office  even  tried 
hard  to  avoid  giving  the  impression  that  they  valued 
one  specific  performance  goal  over  another. 

A.  Requirements  Definition 
and  Management 

LP-3;  The  risks  and  problems  associated  with 
integrating  COTS  into  a  complex  system  were 
underestimated. 

H.  Risk  Assessment/ 
Management 

LP-4;  The  pace  of  the  flight  test  program  was  too  fast 
given  its  cumbersome  mission  plaiming  process  and 
limited  resources.  Test  personnel  were  clearly 
overburdened,  which  appears  to  have  been  a 
contributing  factor  in  the  air  vehicle  3  taxi  mishap. 

E.  Validation  and  Verification 

LP-5;  With  the  major  reduction  in  the  use  of 
specifications  and  standards,  there  was  no 
comprehensive  set  of  requirements  to  judge  that  an 
aircraft  was  safe  to  fly.  This  void  in  the  acquisition 
process  led  to  the  formulation  and  release  of  Air 

Force  Policy  Directive  62-6. 

A.  Requirements  Definition 
and  Management 

C.  System  and  Subsystem 
detailed  design  and 
implementation 

D.  Systems  Interface  and 
Integration 

Miniature  Seeker  Technology  Integration  (Grenville,  Kleiner  and  Newcomb  2004) 


LP-I:  “Build  Porsches,  not  Formula  I’s.  Use  design 
margins  to  reduce  the  level  of  optimizing  at  the  sub¬ 
system  level  and  take  advantage  of  existing  hardware 
architectures. 

B.  System  Architecture  and 
Conceptual  Design 

C.  System  and  Subsystem 
detailed  design  and 
implementation 

LP-2;  Use  daily  meetings  and  an  electronic  problem 
failure  report  approach  to  enable  peer  reviews. 
Embedding  quality  assurance  with  the  team  allowed 
problem  discovery  earlier  and  resolution  earlier  in  the 
process. 

I.  System  and  Program 
Management 

LP-3;  Team  took  ownership  of  the  project.  Each 

I.  System  and  Program 
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responsible  engineering  authority  looked  forward  to 
the  project  horizon.  Team  members  had  more 
responsibility. 

Management 

LP-5;  Keep  the  team  focused  to  accomplish  their 
objectives. 

I.  System  and  Program 
Management 

KC-135  Flight  Training  System  (Chislaghi,  Dyer  and  Free  2010) 


Designing  the  platform  to  be  compatible  with  a 
motion  system  paid  dividends  later  in  the  system’s 
lifecycle  by  providing  a  growth  path  which  facilitated 
the  implementation  of  future  upgrades.  (8) 

B.  Systems  Architecting  and 
Conceptual  Design 

C.  System  and  Subsystem 
detailed  design  and 
implementation 

The  philosophy  employed  by  the  KC-135  aircrew 
training  system  senior  engineering  and  management 
leadership  emphasizes  the  importance  of  open 
communication  lines  between  the  various 
stakeholders.  (14) 

I.  System  and  Program 
Management 

Air  Mobility  Command  has  emphasized  two  key 
program  goals  that  formed  the  foundation  of  the  KC- 
135  aircrew  training  system  upgrade  strategy.  The 
first  addressed  the  need  for  concurrency,  which  is  to 
ensure  the  operational  flight  trainer  is  upgraded  and 
ready  for  training  prior  to  the  aircraft  with  its 
modifications  being  fielded.  The  second  addressed 
the  goal  to  upgrade  operational  flight  simulator 
training  effectiveness.  The  first  goal  emerged  as  a 
result  of  early  successes  in  the  execution  of  the 
simulator’s  upgrade  strategy  concurrent  with  a  major 
aircraft  upgrade  and  modification  program.  (18) 

G.  Life  Cycle  Support 

There  was  no  formal  systems  engineering  process 
followed  for  requirements  allocation.  (36) 

A.  Requirements  Definition 
and  Management 

Added  emphasis  had  to  be  placed  on  managing  risk 
mitigation  in  order  to  ensure  the  right  people  were 
assigned  to  work  the  problem,  mitigation  plans  were 
realistic  and  implementable,  and  that  the  required 
work  was  on  track  to  being  completed  on  schedule. 

(20) 

H.  Risk 

Assessment/Management 

B.  COMMON  SUBMARINE  RADIO  ROOM  SYSTEM  OF  SYSTEMS 
DEVELOPMENT  AND  INTEGRATION  APPROACH 

As  a  system  of  systems,  CSRR  followed  an  established  and  fairly  disciplined 


approach  to  developing  each  version.  The  paradigm  shift  from  simply  building  a 
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collection  of  boxes  to  actively  working  with  various  programs  whieh  were  in  different 
stages  of  maturity  created  many  challenges  and  opportunities.  Throughout  the  entire 
proeess  from  design  and  development  to  sustainment  these  challenges  and  opportunities 
influence  the  CSRR  program  ability  to  deliver  and  support  a  complex  SOS  to  support  the 
submarine  eommunications  requirements. 

1.  Design  and  Development 

The  initial  version  of  CSRR  was  based  on  a  contraetor-furnished  design  for  the 
VA  class.  Following  a  failure  by  the  subeontractor  to  deliver  a  system  to  the  shipbuilder 
PEO  SUB  performed  an  analysis  of  alternatives  (AOA).  The  AOA  reeommended  several 
options  (PMW770  2008,  11). 

1.  Sole  souree  contract  to  Electric  Boat  and  Eockheed  Martin-Maritime 
Systems  and  Sensors 

2.  A  full  and  open  competition  and  a  government-industry  team  development 
and  production  effort 

3.  Government  industry  team 

Option  three  was  ehosen  to  support  the  CSRR  work  for  the  SSGN.  Work  from  the 
SSGN  development  was  carried  forward  and  leveraged  to  support  OPNAV’s  direction  to 
design  a  CSRR  variant  for  the  SSBN.  The  main  requirement  of  CSRR  is  to  integrate 
other  PORs.  Several  PORs,  such  as  ADNS,  were  not  ready  in  time  to  support  the  delivery 
in  support  of  the  development  efforts  so  PMW770  developed  suitable  replacement 
solutions  to  support  program  delivery.  This  solution  enabled  CSRR  to  effectively  work  as 
a  directed  SOS  up  through  CSRR  V2.  Prom  an  SOS  perspective  CSRR  could  have  been 
defined  as  a  direeted  SOS.  A  benefit  of  being  a  directed  SOS  is  CSRR  had  a  large  degree 
of  eontrol  over  design,  configuration  management  and  sustainment.  The  disadvantage  is  a 
solution  whieh  increased  overall  total  ownership  costs  (TOC)  to  the  CSRR  program. 

The  original  approach  for  developing  eaeh  version  of  CSRR  was  related  to  a 
specifie  submarine  class.  The  SSGN  VO  leveraged  the  VA  VO  design.  The  SSBN  and  SW 
used  the  SSGN  VO  design  as  a  basis  for  their  development.  Each  of  these  designs  were 
built  and  completed  a  full  system  design  verification  test  (SDVT)  and  systems  acceptanee 
test  (SAT)  prior  to  their  introduction  to  the  fleet.  SSBN  VI  began  the  next  cycle  with 
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SSGN  and  SW  V2  closely  following  behind.  In  eaeh  ease  a  full  design  was  again  built 
and  fully  tested  to  validate  the  design  and  verify  functionality.  For  these  versions  it  was 
not  diffieult  to  maintain  a  single  design  sinee  only  four  SSGNs,  three  SW  and  14  SSBNs 
were  operational.  The  changes  to  the  VA  design  were  ineorporated  with  the  shipbuilder 
to  deliver  a  minimal  eapability  and  upgrades  oecurred  as  eaeh  platform  entered  a  post 
shakedown  availability  (PSA).  Following  PSA  responsibility  of  the  VA  CSRR  shifted  to 
PMW770. 

The  development  of  V2  for  SSGN  and  SW  attempted  to  leverage  available  PORs, 
but  one  of  the  initial  problems  noted  is  their  equipment  had  not  eompleted  their  own 
testing,  was  planned  to  oecur  concurrently,  or  it  was  aecomplished  without  eonsulting 
with  the  CSRR  team  to  assure  their  approaeh  would  work.  The  maturity  of  two  systems 
fell  behind  despite  assuranees  from  the  POR  they  would  be  ready.  The  first  was 
diseovered  at  the  SSGN  CDR  whieh  foreed  a  signifieant  design  ehange  to  remove  all 
interfaee  eabling  when  it  failed  to  deliver  a  system.  Another  issue  with  this  program 
resulted  from  the  CSRR  team  reeonfiguring  the  system  in  order  to  physieally  fit  the 
eomponents  in  the  design.  The  program  had  eompleted  their  design  environmental  testing 
without  engaging  the  CSRR  team  to  assess  its  ability  to  fit  in  the  design.  This  resulted  in 
invalidating  the  environmental  testing  when  the  eomponents  were  reloeated  to  fit  in  the 
radio  room.  The  other  oeeurred  when  the  POR  reported  their  software  would  be  delayed 
one  year.  This  was  identified  just  prior  the  beginning  of  the  installation.  The  CSRR 
program  had  agreed  to  proeure  the  hardware  but  the  laek  of  software  foreed  development 
of  a  temporary  solution. 

A  similar  issue  oeeurred  with  the  SSBN  VI  when  modifieations  to  an  antenna 
system  did  not  eomplete  all  of  their  design  work  in  time  to  support  the  seheduled 
modernization  period.  While  the  CSRR  and  antenna  modernization  were  related  but 
separate  efforts  it  still  represented  a  laek  of  synehronization  of  activities.  Several 
agreements  were  established  with  ADNS  and  other  programs  to  arrange  a  shift  of 
sustainment  responsibilities  and  to  agree  upon  the  delivery  of  POR  systems  in  the  future. 

At  this  time,  the  LA  elass,  originally  deferred  from  FYIO  to  FY15,  were 

aeeelerated  to  reaeh  a  planned  IOC  in  FYI2.  Sinee  the  LA  elass  eonstituted  the  majority 
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of  the  submarines  in  the  force  and  was  already  facing  obsolescence  issues,  it  was  selected 
to  be  the  first  V3  platform.  In  2009,  the  CSRR  V3  preliminary  design  review  (PDR) 
initiated  a  transition  from  an  informal  directed  SOS  to  a  recognized  acknowledged  SOS 
utilizing  other  POR  systems  to  deliver  capability.  A  benefit  of  this  approach  more  closely 
aligned  CSRR  development  with  the  direction  of  the  program  acquisition  plan  (AP)  to 
fully  leverage  other  POR  systems.  One  advantage  of  this  shift  in  approach  is  the 
reduction  of  TOC  for  the  CSRR  program.  Reduction  of  overall  SOS  TOC  is  questionable 
since  these  costs  were  spread  across  a  number  of  programs.  Another  advantage  is  the 
CSRR  team  no  longer  had  to  identify  and  procure  these  components.  The  disadvantage  is 
the  number  of  configuration  management  and  logistics  challenges  increased  with  each 
new  POR  as  they  introduced  changes  in  their  systems.  These  changes  had  to  be  assessed 
for  their  impact  to  the  overall  SOS  design.  Using  the  other  PORs  in  many  cases  identified 
their  designs  had  to  be  modified  in  order  to  be  accommodated  physically  and 
functionally.  Changes  to  the  system  designs  had  to  be  negotiated  with  the  POR  which  in 
turn  impacted  cost  and  schedule  (Steve  Devin  2014,  email  questionnaire).  In  one  case 
SDVT  had  to  be  halted  to  identify  the  source  of  heating  issues  in  the  inboard  racks.  The 
issue  was  resolved  through  reversing  component  fans,  fixing  cooling  shorts  and 
redirecting  more  cooling  to  the  radio  room.  The  outcome  of  this  approach  identified  the 
need  to  engage  with  the  PORs  earlier  to  ensure  any  submarine  unique  requirements  were 
included  in  their  documentation. 

Another  issue  was  identified  just  prior  to  beginning  the  first  installation  impacted 
the  certification  of  the  messaging  system.  The  change  in  its  certification  forced  a  change 
how  information  could  be  routed.  A  solution  was  identified  but  was  not  installed  on  the 
first  platform.  The  increased  density  of  cables  and  network  components  in  the  radio  room 
created  cable  management  issues  which  increased  the  difficulty  of  racking  out  equipment. 
This  was  first  noted  by  the  production  team  as  they  assembled  the  production  kits  in  their 
facility  for  pre-installation  testing  and  checkout  (PITCO).  The  issues  were  confirmed  on 
the  first  several  platforms  which  in  turn  forced  a  redesign  effort  to  strengthen  cable 
retractors  and  reroute  cabling.  The  number  of  issues  found  confirmed  more  maturation  of 
POR  components  was  needed  prior  to  beginning  integration  into  the  CSRR  architecture. 
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Additionally  closer  coordination  between  the  design  and  production  team  needs  to  occur 
in  order  to  identify  issues  impacting  production,  installation  and  maintainability. 

Effectively  managing  an  SOS  program  sueh  as  CSRR  requires  ereating  and 
maintaining  effective  teams.  The  NUWC  design  team,  shown  in  Figure  24,  is  responsible 
to  support  the  design,  integration  and  testing  to  ensure  CSRR  and  each  of  the  PORs 
continued  to  meet  their  performance  requirements.  The  lead  systems  engineer,  also  the 
systems  architect,  coordinates  the  efforts  of  the  platform  engineers  to  maintain  the 
eommonality  of  the  CSRR  design  aeross  all  platforms  while  balancing  the  needs  of  the 
individual  programs  and  platforms.  The  chief  engineer  in  essence  devotes  much  of  his 
efforts  to  “herding  the  cats”  toward  a  common  goal.  When  interviewed  the  CSRR  chief 
engineer  (Mike  Gozzo  2013  personal  eommunieation)  defined  his  roles  and 
responsibilities  as  follows: 

1 .  Leading  a  team  of  engineers  throughout  all  aspeets  of  the  systems 
engineering  process.  This  includes: 

2.  Developing  plans  and  processes  to  aehieve  the  desired  program 
goals  and  monitor  progress  towards  those  goals. 

3.  Collaborate  with  technical  experts  to  outline  the  overall 
architecture  and  design  of  baseline  modernization. 

4.  Be  knowledgeable  enough  in  all  areas  of  the  system  to  be  able  to 
discern  important  issues  vice  minor  concerns  or  wants. 

5.  Constantly  wateh  for  feedbaek  of  failing  or  inadequate  proeesses 
and  implement  course  correetions. 

6.  Act  as  final  decision  for  technical  and  non- technical  issues  as 
required.  (Mike  Gozzo  2013  personal  eommunieation) 

The  design  agent  works  with  the  ehief  engineer  who  oversees  the  design  team. 
The  design  team  is  composed  of  platform  engineers  who  are  supported  by  functional 
SMEs  from  other  areas  such  as  network  systems,  software,  information  assurance, 
integrated  logistics,  testing  and  evaluation  and  others.  The  SMEs  from  the  other  programs 
are  available  to  provide  information  and  expertise  during  the  integration  SOS  activities. 
The  challenge  of  working  with  other  programs  is  identifying  the  appropriate  definition 
point  between  what  is  entirely  within  a  FOR  responsibility  and  what  impacts  the  SOS  as 
a  whole.  Eaeh  platform  engineer  is  responsible  for  creating  and  traeking  the  baseline  for 
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the  version  planned.  These  results  are  shared  within  the  loeal  design  teams  so  maximize 
the  sharing  of  information  between  the  platform  engineers.  The  results  produce  the 
information  necessary  to  support  installing  a  version  of  CSRR. 

The  chief  engineer  must  have  a  vision  of  what  the  overall  SOS  is  going  to  be  in 
terms  of  capability,  function,  and  physical  design  (Mike  Gozzo  2013  personal 
communication).  He  describes  these  as 

1 .  Understanding  the  inter-relationships  of  the  subsystems  and  the 
components.  These  drive  the  end  to  end  capabilities  and  determine  if  they 
can  be  achieved  (Mike  Gozzo  2013  personal  communication). 

2.  Look  at  the  functional  block  diagrams  early  in  the  process  to  determine  if 
the  proposed  design  blocks  will  work  together?  The  next  level  looks 
beyond  functionality  of  the  interrelationships  down  to  the  physical  and 
logical  relationships  (Mike  Gozzo  2013  personal  communication). 

The  CSRR  program  envisioned  using  an  evolutionary  approach  for  the 
development  of  each  version.  Each  version  would  be  developed  based  on  the  expected 
capabilities  needed.  Version  zero  replaced  the  legacy  architecture  and  most  of  the 
components.  Version  one  delivered  improvements  to  RFDACS  and  the  operator 
workstations.  Version  two  provided  improvements  to  network  components  and 
introduced  SHF.  Version  three  was  originally  planned  to  include  JTRS  and  ADNS 
Increment  three  as  the  major  capability  drivers.  As  each  version  is  created  the  work  from 
the  previous  one  is  leveraged  to  maximize  the  open  architecture  and  commonality  in 
terms  of  hardware  and  software. 
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Figure  24.  NUWC  Design  Team  Organization  (after  Anderson  2014) 


The  challenge  of  developing  a  version  lies  with  the  expected  maturity  of  the 
systems  planned  for  integration  (Steve  Devin  2014,  email  questionnaire).  Deployed 
systems  are  usually  mature  and  may  be  difficult  and  expensive  to  change.  Systems  still  in 
the  early  stages  of  development  introduce  added  risk  through  additional  changes  arriving 
late  in  the  development  cycle.  Attempting  to  use  the  SOS  engineering  and  integration 
process  to  force  maturation  introduces  potential  rework  and  testing,  and  potential 
recertification  which  can  impact  cost,  schedule  and  performance  (Steve  Devin  2014, 
email  questionnaire).  Finding  the  proper  balance  is  a  constant  challenge  for  the  lead 
systems  engineer.  Figure  25  (DAU  2013,  37;  Dahmann  et  al  2011)  reflects  the  SOS 
system  engineers  view  to  implementing  an  SOS.  The  wave  model  begins  with  each 
incremental  change  planned  for  the  overall  SOS.  The  wave  model  represents  an  iterative 
process  to  analyze,  design,  build,  test  and  deploy  an  SOS.  Common  Submarine  Radio 
Room  has  documented  their  process  in  a  value  stream  analysis  (VS A)  which  enables  a 
great  deal  of  repeatability. 
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External  Environment 


Figure  25.  Application  of  the  Wave  model  to  CSRR 


Application  of  the  wave  model  to  CSRR  is  performed  in  concert  with  the  system 
engineering  technical  review  (SETR)  process  from  the  acquisition  guidance  (DOD, 
2013).  Initiating  the  SOS  design  begins  with  the  SOS  analysis  which,  depending  on  the 
maturity  of  the  proposed  systems  is  normally  an  initial  technical  review  or  a  PDR.  The 
PDR  outlines  the  functional  baseline  of  the  systems  involved  and  assessing  the  proposed 
changes  to  the  SOS  baseline.  Once  the  proposed  changes  are  agreed  upon  detailed 
planning  of  the  SOS  update  takes  place  and  is  reviewed  again  at  the  CDR.  Approval  at 
CDR  establishes  the  physical  architecture  of  the  SOS  leading  to  completion  of  the 
development,  integration  and  certification  of  the  SOS  architecture.  Following 
certification  the  SOS  architecture  will  be  implemented  as  an  update.  Further  changes 
leverage  the  previous  development  cycle.  This  process  occurs  during  the  development  of 
each  version  of  CSRR  as  systems  and  capabilities  are  inserted  and  removed.  This  same 
process  can  be  applied  to  other  related  SOS  such  as  SWFTS,  TBMCS,  or  a  VA  platform. 

Enclosure  three  of  interim  DOD  instruction  5000.02  Operation  of  the  Defense 
Acquisition  System  (DOD  2013,  82)  mandates  the  use  of  SE  principles  as  part  of  the 
acquisition  activities.  The  SETR  process  defines  the  mandatory  and  recommended 
activities  that  occur  over  a  programs  life  cycle.  The  challenge  with  this  is  the  SETR 
process  was  created  to  address  a  single  program.  The  DOD  instruction  5000.02  addresses 
SOS  briefly  in  terms  of  establishing  a  special  interest  program,  developing  an  acquisition 
strategy,  identifying  a  lead  systems  integrator  and  testing.  If  a  program  happens  to  be  a 
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directed  SOS  most  of  these  activities  are  occurring  as  needed  in  the  overall  effort.  Since 
many  mission  capabilities  are  the  result  of  creating  an  acknowledged  SOS  the  individual 
programs  may  be  in  different  phases  of  their  life  cycle.  The  main  challenge  to  the  SOS 
engineer  is  coordinating  the  integration  and  implementation  of  changes  to  minimize 
potential  negative  impacts.  DASN  (RDT&E)  drafted  a  revision  three  to  the  current  Naval 
System  of  Systems  Engineering  Guidebook  (ASN  (RDA)  2006).  The  focus  of  the  revision 
is  described  in  the  foreword  of  the  draft 

The  focus  of  this  Guidebook  is  on  the  mission  level  System  of  Systems 
engineering  process  to  provide  needed  capabilities  and  functionality 
within  a  Net  Centric  Operations  and  Warfare  environment  in  support  of  a 
Mission  Area  Capability.  It  provides  a  guide  to  recommended  processes, 
methods  and  tools  that,  when  applied  by  the  Mission  Area  Systems 
Engineers,  will  aid  program  managers,  their  SEIPTs,  support  teams,  and 
contractors  in  producing  systems  that  successfully  deliver  the  Mission 
Area  capability.  (DASN  (RDT&E)  2013) 

Revision  three  significantly  revises  the  content  but  the  end  goal  of  delivering 
mission  capability  from  a  SOS  context  is  preserved.  An  important  distinction  in  revision 
three  is  how  a  SOS  is  redefined  in  the  context  of  mission  capability.  The  following  quote 
from  the  background  captures  this  new  definition  and  intent. 

In  the  future,  global  operations  will  be  conducted  by  distributed,  integrated 
and  interoperable  forces.  This  future  warfare  is  about  capability  delivered 
by  a  “SOS”  operating  as  a  single  system.  SOS  is  defined  in  this  document 
as  a  force  package  of  interoperable  platforms  and  nodes  acting  as  a  single 
system  to  achieve  a  mission  capability,  i.e.  a  mission  level  SOS.  Typical 
characteristics  include  a  high  degree  of  collaboration  and  coordination, 
flexible  addition  or  removal  of  component  systems,  and  a  net-centric 
architecture.  The  capabilities  provided  by  each  constituent  system 
operating  within  the  SOS  are  framed  by  the  integrated  force  package 
architecture.  (DASN  (RDT&E)  2013,  6) 

In  order  to  achieve  these  capabilities  the  SETR  processes  must  be  applied  at  each 
level  (e.g.,  the  CSRR  SOS  must  support  the  platform  and  mission  level  SOS).  The  PORs 
supporting  CSRR  follow  the  SETR  processes  as  part  of  their  development  cycle.  The 
level  of  complexity  and  amount  of  change  within  each  system  and  the  maturity  of  the 
program  can  determine  which  SETR  events  may  be  included  or  omitted.  A  SOS  will  be 
using  an  iterative  process  previously  described  and  may  include  events  such  as  an  initial 
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technical  review,  systems  requirements  review,  or  software  specification  reviews.  The 
CSRR  value  stream  analysis  identified  the  following  SETR  events  in  Table  18  which 
occur  during  the  design  and  development  phase.  The  SETR  events  are  not  the  main  goals 
within  a  program  development  cycle  but  status  reviews  assessing  the  maturity  of  a  system 
or  SOS  to  progress  to  the  next  phase. 

Erom  a  SOS  approach  each  individual  system  performs  these  SETR  events  as 
well.  These  FOR  SETR  events  are  important  to  the  SOS  since  they  drive  maturity  and 
demonstrate  they  are  ready  to  be  implemented  in  the  overall  architecture.  The  design- 
build  approach  at  both  the  systems  level  and  the  SOS  level  in  concert  with  the  SETR 
events  attempt  to  minimize  the  risks  of  introducing  a  capability  before  it  is  ready. 


Table  18.  System  Engineering  Events  Occurring  During  Design  and 

Development 


Event 

Activity 

Preliminary  Design  Review 

(FDR) 

The  FDR  provides  the  initial  review  of  the  design  at  the 
functional  level.  At  this  point  individual  systems  maturity 
is  still  low,  approximately  20-30  percent. 

Integrated  Baseline  Review 

(IBR) 

The  IBR  follows  the  PDR  to  establish  the  development 
baseline.  Each  development  baseline  is  based  on  a  Lean 

Six  Sigma  (ESS)  Value  Stream  Analysis  (VSA)  which 
identified  the  major  events  and  the  time  required  to 
accomplish  them.  The  VSA  uses  an  accordion  concept 
allowing  each  version  development  baseline  to  expand  or 
shrink  based  on  the  expected  amount  of  work.  Once  the 
development  baseline  is  approved  it  provides  the  main 
tracking  method  to  assess  if  schedule  was  being 
maintained. 

Critical  Design  Review 

(CDR) 

CDR  is  scheduled  to  occur  when  the  design  maturity  is 
estimated  to  be  approximately  85-90  percent.  At  this  point 
the  physical  design  is  presented  along  with  proposed 
testing  criteria  and  assessments  from  the  production,  ILS 
and  lA  leads.  Systems  that  are  not  determined  to  be  mature 
by  CDR  are  recommended  for  deferral  to  the  next  version. 
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2,  Testing  and  Certification 

CSRR  follows  a  build-test-certify  approach  during  the  development  eycle  as  a 
risk  reduction  methodology.  Onee  a  component  or  system  is  reeeived  the  CSRR  team 
performs  several  levels  of  testing  prior  to  deployment  to  the  fleet.  The  goal  is  to  minimize 
repeating  any  testing  performed  by  the  parent  FOR  while  ensuring  it  will  fit  and  funetion 
within  the  CSRR  arehiteeture  and  will  be  interoperable.  Figure  26  illustrates  the  systems 
whieh  undergo  some  level  of  testing  within  the  overall  system  of  systems. 

With  the  exeeption  of  funetional  interfaee  testing  and  regression  testing  a  test 
readiness  review  will  oeeur  to  get  coneurrence  from  the  principal  assistant  program 
manager  (PAPM)  or  program  manage  the  CSRR  is  ready  to  begin  testing.  The  levels  of 
testing  are  deseribed  below.  Onee  the  testing  is  eomplete  the  results  are  shared  with  the 
stakeholders  as  neeessary  to  support  meeting  program  aequisition  milestones. 


Figure  26.  Systems  Testing  Required  in  Support  of  Common  Submarine  Radio 

Room  as  a  System  of  Systems 
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a.  Functional  Interface  Testing 

This  is  the  first  time  the  system  or  component  can  be  actually  evaluated  by  the 
design  team.  The  system  is  evaluated  for  form  and  fit,  logical  and  power  interfaces  are 
checked,  and  the  system  documentation  is  reviewed.  Accomplishment  of  this  testing  is 
under  the  direction  of  the  design  agent  and  chief  systems  engineer.  Once  this  is  complete, 
the  system  or  component  is  reported  at  the  CDR  as  ready  to  support  SDVT. 

b.  System  Design  Verification  Testing 

SDVT  is  a  formal  test  event  to  validate  the  system  will  operate  within  the  CSRR 
architecture  using  an  end  to  end  environment,  will  meet  its  own  system  requirements,  and 
will  not  degrade  the  CSRR  performance  specifications  defined  in  the  SUBECS  CPD. 
When  the  system  is  ready  to  enter  SDVT  the  design  agent  and  test  and  evaluation  lead 
will  brief  the  PAPM  and  request  concurrence  to  begin  testing.  Once  the  system  has 
successfully  completed  SDVT,  the  physical  design  has  been  fully  verified  and  validated. 
If  necessary  any  regression  testing  required  will  occur  after  SDVT.  If  this  is  a  first  of 
version  design,  the  CSRR  will  proceed  into  SAT. 

c.  Systems  Acceptance  Testing 

Systems  acceptance  testing  is  performed  if  the  design  is  a  first  of  version  or  has 
been  determined  necessary  by  the  program  team.  Systems  acceptance  testing  is  the  final 
performance  run  to  demonstrate  system  stability  while  operating  during  a  series  of 
scenarios.  Operating  procedures  are  validated  using  fleet  operators  and  system 
configuration  information  is  collected  to  support  development  of  the  CSRR  Multi- 
Reconfigurable  Training  System  (MRTS).  If  necessary  any  strategic  certification  testing 
will  occur  as  well  as  collecting  equipment  data  to  determine  operational  availability  (Aq). 
Testing  is  accomplished  using  operational  circuits  with  the  submarine  BCA.  Authority  to 
proceed  into  SAT  is  granted  by  the  PMW770  program  manager  at  the  SAT  test  readiness 
review. 
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d.  Regression  Testing 


Any  deficiencies  or  changes  that  occur  during  SDVT  or  SAT  require  some  level 
of  retest  to  verify  if  a  patch  or  configuration  change  works.  Regression  testing  is  not 
considered  a  required  event  but  enough  time  is  normally  scheduled  between  SDVT  and 
SAT  testing  of  any  changes. 

e.  Cybersecurity  Testing 

Cybersecurity  (or  lA)  testing  will  take  place  in  concert  with  the  formal  testing 
events  and  as  necessary  in  between  testing  events  to  verify  security  technical 
implementation  guides  are  applied  and  validate  they  are  working.  Any  updates  that  need 
to  be  installed  will  be  accomplished  prior  to  SDVT  and  SAT. 

/.  Strategic  Certification  Testing 

Strategic  testing  is  required  by  the  Joint  Staff  to  ensure  any  changes  to  a  NC3 
system  have  been  properly  and  adequately  verified  and  validated  prior  to  deployment  to 
an  operational  NC3  activity  or  platform.  The  primary  certification  tests  are  TCM  and 
EAM  certification. 

1.  TCM  Certification — TCM  certification  is  accomplished  to  validate  any 
hardware  or  software  changes  made  to  the  messaging  path  by  transmitting 
a  predetermined  number  of  targeting  messages  and  recording  them  to 
media.  An  agent  for  SSP  analyzes  the  messages  to  verify  they  are  fully 
readable  by  the  strategic  weapons  system.  Any  discrepancies  are  analyzed 
and  corrected  before  issuing  the  final  certification  recommendation  to  U.S. 
Strategic  Command. 

2.  EAM  Certification — Clear  and  concise  communications  between  the 
president  and  strategic  forces  requires  the  use  of  highly  reliable 
communications  paths.  In  accordance  with  JCS  direction  any  changes  to 
systems  impacting  the  messaging  paths  are  tested  prior  to  being  fielded. 
The  certification  is  an  end  to  end  test  to  verify  any  changes  have  not 
impacted  the  reliable  delivery  of  E AMs.  Testing  results  are  provided  to  the 
JCS  for  review  and  approval. 

The  design  and  development  process  maintains  the  rigor  necessary  to  ensure  any 
issues  or  deficiencies  are  resolved  or  mitigated  before  deployment  in  an  operational 
environment.  The  challenge  from  the  system  versus  SOS  perspectives  is  defining  what 

the  right  level  of  design,  development  and  testing  that  must  be  done.  The  design  and 
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development  activities  are  largely  driven  by  physical  and  functional  characteristics  of 
each  system.  Testing  becomes  a  more  contentious  issue  at  times,  especially  if  the  FOR 
feels  they  have  performed  an  adequate  level  of  testing  to  demonstrate  they  are  ready  for 
fielding.  History  within  DOD  is  replete  with  examples  where  this  argument  has  been 
proven  false.  TBMCS  is  one  example  where  unclear  requirements  and  over  reliance  on 
the  contractor  led  to  being  unable  to  validate  or  verify  the  operational  readiness  prior  to 
deployment  (Cohens  and  Krause  2005,  v,  27-37).  The  Hubble  space  telescope  is  another 
where  the  system  was  launched  into  orbit  before  a  flaw  in  the  main  mirror  was  detected. 
The  repair  required  an  unplanned  1 1  day  space  mission  by  the  shuttle  Endeavor  (Mattice 
2003,  10).  The  F-111  attempted  to  implement  concurrency  of  design  validation  and 
verification  while  entering  production.  The  validation  and  verification  resulted  in  several 
costly  design  changes  to  200  aircraft  and  schedule  delays  due  to  structural  failures  which 
grounded  the  entire  F-111  fleet  for  several  months  (Richey  2005,  24).  While  it  can  be 
extremely  challenging  to  test  every  possible  scenario  the  testing  approach  from  the 
system  to  the  SOS  should  follow  a  logical  progression  to  maintain  traceability  and 
identify  potential  areas  of  risk  to  investigate  further.  It  is  incumbent  on  the  SOS  program 
manager  to  keep  the  stakeholders  aware  of  the  status  and  concerns  of  any  shortfalls  in  the 
testing.  Vaneman  and  Budka  (2013)  illustrated  the  role  of  SOSE  integration  in  Figure  27. 
Each  FOR  performs  the  activities  shown  in  the  lower  portion  of  the  Vee.  If  these  are  not 
adequately  performed  or  incomplete,  the  validation  and  verification  necessary  for 
certification,  deployment  and  sustainment,  shown  in  Figure  28,  increase  the  risk  of 
failure.  This  approach  is  also  reflected  in  table  A-6  of  annex  A  in  the  SE  Guide  for  SOS 
(Director,  Systems  and  Software  Engineering  2008,  103)  of  the  importance  of  the  SOS 
verification  building  on  the  efforts  of  the  individual  systems.  Once  the  CSRR  version  has 
completed  the  design  and  development  phase,  responsibility  for  procuring  the  materials 
and  equipment  shifts  to  the  production  agent. 
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Figure  27.  System  of  Systems  Engineering  and  Integrations  Role  in  System 
Design  and  Development  (from  Vaneman  and  Budka  2013,  6) 
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Figure  28.  Validation  and  Verification  Supporting  the  System  of  Systems 
Interoperability,  Deployment  and  Sustainment  (after  Vaneman  and 

Budka  2013,  6,  11) 


The  design  agent  also  manages  responsibilities  as  the  CSRR  planning  yard  to 
maintain  configuration  control  of  each  CSRR  version.  The  purpose  of  the  planning  yard 
is  to  serve  as  the  repository  for  all  drawings  concerning  CSRR.  The  CSRR  planning  yard 

maintains  a  partnership  with  the  platform  planning  yards  to  manage  changes  that  occur 
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within  the  CSRR  boundary  and  the  platform  boundaries.  Changes  are  managed  via  a 
planning  yard  liaison  action  request  (PLAR)  between  the  design,  production  and 
installation  teams.  Any  changes  that  occur  post  testing  will  be  assessed  to  determine  the 
impact  and  if  any  additional  design  or  testing  is  required. 

C.  COMMON  SUBMARINE  RADIO  ROOM  SYSTEM  OF  SYSTEMS 

PRODUCTION  APPROACH 

Prior  to  CSRR,  SCSS  modernization  involved  the  coordination  of  multiple 
programs  to  build  and  ship  their  own  installation  kits.  This  meant  the  alteration 
installation  team  had  to  verify  they  had  all  of  the  right  materials  and  equipment  and 
develop  their  own  integrated  drawings  to  accomplish  the  installation.  Lack  of 
standardization  between  the  organizations  and  programs  created  a  significant  amount  of 
variation  of  what  a  kit  would  contain.  This  approach  also  created  complications  if  several 
install  kits  were  managed  by  different  installation  teams.  The  lack  of  coordination  added 
complexity  in  a  very  dynamic,  fully  scheduled  availability.  This  approach  also  created  a 
substantial  amount  of  rework  if  errors  were  found  in  a  ship  alt  package  or  no  guidance 
was  provided  for  configuration  issues.  Ultimately,  this  approach  ended  up  creating  48 
similar  yet  different  configurations  among  the  LA  platforms.  This  same  approach  has 
also  resulted  in  creating  different  configurations  among  the  several  hundred  surface 
platforms  as  well. 

The  SSC  LANT  production  agent  was  interviewed  as  part  of  this  research  via 
email  and  telephone.  SSC  LANT  oversees  production  activities  to  include  procurement  of 
all  equipment  and  materials  necessary  to  support  a  CSRR  installation,  PITCO,  kitting, 
and  shipment  to  the  site  (Dave  Bednarczyk  telephone  interview  2014).  By  leveraging 
opportunities  for  quantity  purchases  of  installation  materials,  significant  cost  savings  can 
be  realized.  Individual  PORs  provide  their  equipment  and  any  unique  materials  necessary 
for  inclusion  in  the  installation  kit.  The  production  team  shown  in  Figure  29  engages  with 
the  various  PORs  as  necessary  to  coordinate  the  procurement  or  delivery  of  their 
equipment  and  materials  for  PITCO  and  shipping  to  the  installation  team.  The  production 
team  assigned  responsibilities  by  platform  to  address  specific  issues  while  balancing 

workloads.  The  PITCO  period  allows  pre-loading  and  configuring  software  and 
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hardware.  Additionally  it  serves  as  a  burn  in  period  to  eliminate  possible  failures  prior  to 
shipping.  Onee  PITCO  is  complete,  the  installation  kit  is  packed  and  shipped  to  the 
installation  site.  Similar  to  the  design  agent’s  role  in  performing  SOS  verification,  PITCO 
provides  the  overarching  testing  to  ensure  the  product  shipped  to  the  site  is  operational. 
This  production  quality  assurance  approach  mitigates  the  risk  of  failures  occurring  during 
the  production  and  systems  operational  verification  testing  (SOVT)  (Dave  Bednarczyk 
telephone  interview  2014). 

The  PITCO  process  successfully  demonstrated  for  VI  and  V2  proved  to  be  more 
difficult  to  perform  for  V3.  Some  PORs  chose  to  ship  their  systems  directly  to  the 
installation  team  while  others  sent  them  to  SSC  LANT  without  their  final  configuration 
settings  inserted.  Having  some  incomplete  components  and  others  not  available  impacted 
the  ability  to  perform  a  complete  PITCO  and  validate  the  ship  set  prior  to  packing  and 
shipping. 


Figure  29.  CSRR  Production  and  Support  Team 
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Late  changes  to  software  and  hardware  configurations  also  caused  delays.  The 
initial  V3  ship  set  installed  on  the  USS  Hampton  did  not  perform  PITCO  which  resulted 
in  a  number  of  failures  caught  by  the  installation  and  SOVT  teams.  Failed  parts  had  to  be 
replaced  from  the  CSRR  production  team  and  the  other  programs.  These  failures  in  turn 
drove  up  installation  costs  and  delayed  schedules.  Data  collected  from  follow  on 
installations  by  the  production  agent  confirmed  failure  to  accomplish  a  PITCO  continued 
to  drive  costs  upwards  of  several  hundred  thousand  dollars  and  delay  completion  from 
days  to  weeks  (Dave  Bednarczyk  telephone  interview). 

Previous  to  the  deployment  of  V3  the  production  team  procured  and  managed  all 
of  the  system  components  necessary  to  install  CSRR.  As  the  procurement  arm  for  the 
CSRR  program  SSC  LANT  was  able  to  control  their  fate  by  purchasing  or  designing  the 
materials  necessary  to  build  a  ship  set  (Dave  Bednarczyk  telephone  interview).  The 
advantage  of  this  approach  is  production  and  kitting  responsibilities  solely  resided  with 
the  production  agent.  The  disadvantage  of  this  is  systems  normally  managed  by  another 
program  office  used  a  different  configuration  and  could  not  easily  assume  responsibility 
of  these  new  components  within  their  budgets.  Specific  components,  such  as  the  EHF 
Follow  on  Terminal,  was  provided  from  PMW  170.  Others  had  to  be  designed  entirely  to 
fulfill  requirements  if  a  formal  POR  was  not  available.  One  component  which  was 
entirely  built  to  support  CSRR  was  the  modern  legacy  cryptographic  system  (MFCS) 
which  was  originally  planned  as  a  replacement  for  the  multifunctional  cryptographic 
system  (MCS).  The  vendor’s  inability  to  prove  the  viability  of  the  MCS  resulted  in  its 
cancellation  in  2004.  In  order  to  keep  the  CSRR  program  on  track  an  alternate  solution 
was  rapidly  developed  and  deployed  with  version  zero.  The  MFCS  was  created  in  less 
than  a  year  to  provide  the  similar  capabilities  as  the  MCS.  Created  only  as  a  stopgap 
solution,  the  MFCS  is  being  replaced  with  a  POR  crypto  universal  enclosure  managed  by 
PMW130. 

Version  three  shifted  to  an  approach  of  using  consolidated  engineering  changes 
(EC)  and  ship  alteration  record  (SAR)  with  the  ship  installation  drawings  (SID)  contained 
in  an  associated  EC  creating  what  was  called  a  “SIDless  SAR.”  The  creation  of  the  ECs 
and  their  associated  SARs  were  used  to  create  the  consolidated  list  of  bill  of  materials  the 
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production  team  would  provide  and  a  list  the  installing  aetivity  would  have  to  provide. 
This  approaeh  identified  a  number  of  issues  when  engaging  with  other  PORs.  For  V3, 
PORs  were  expeeted  to  provide  their  equipment  and  either  the  installation  materials  or 
funding  to  proeure  them.  This  approaeh  added  another  layer  of  eoordination  whieh 
eaused  eonfusion,  resulting  in  equipment  reeeived  with  the  wrong  eonfigurations, 
damaged,  or  shipped  late.  As  a  build  to  print  organization,  introdueing  late  ehanges  to  the 
SSC  LANT  produetion  team  eaused  perturbations  in  eosts  and  sehedules  due  to  rework 
(Dave  Bednarezyk  telephone  interview).  PLARs  direeting  ehanges  to  eables  or  mounting 
kits  often  meant  pulling  materials  out  of  a  paeked  kit,  ereating  a  risk  of  something  being 
misplaeed.  This  proeess  is  refieetive  of  an  aeknowledged  SOS  oharaeteristie  where  the 
SOS  has  objeetives,  resourees  and  manager  but  must  also  eollaborate  with  the  eonstituent 
systems  (Direetor,  Systems  and  Software  Engineering  2008,  5). 

D,  COMMON  SUBMARINE  RADIO  ROOM  SYSTEM  OF  SYSTEMS 

INSTALLATION  APPROACH 

Common  Submarine  Radio  Room  installations  are  performed  by  alteration 
installation  teams  (AIT)  eontraeted  through  the  SPAWAR  Systems  Center  Installation 
Management  Offiee  (IMO).  In  2011,  the  FRD  was  established  to  provide  a  single  agent 
responsible  for  eoordinating  installations.  The  platform  installation  manager  (PIM)  is  the 
embedded  FRD  representative  within  the  Undersea  Integration  program  offiee 
responsible  for  eoordinating  installations.  The  PIM  works  with  the  respeetive  produet 
program  offiees  to  ensure  the  equipment,  materials  and  personnel  are  available  to  support 
the  installation. 

The  installation  paekages  developed  by  the  design  agent  ineluded  the  bill  of 
materials  needed  to  aeeomplish  the  work.  The  intent  was  to  have  the  POR  fund  their 
share  and  leverage  the  advantages  of  making  quantity  purehases.  This  effort  resulted  in 
an  agreement  between  PMW770  and  several  program  offiees  in  whieh  the  CSRR 
program  would  purehase  the  materials  and  deliver  them  in  the  kit.  The  other  PORs 
installation  funding  would  be  used  first  and  then  CSRR  installation  funding  would  be 
used. 
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Each  installation  is  assigned  government  onsite  installation  eoordinator  (OSIC)  to 
serve  as  the  liaison  between  the  platform,  the  loeal  maintenanee  aetivities,  and  the 
installation  team.  The  OSIC  is  responsible  for  arranging  for  the  modernization,  testing, 
and  training  aetivities.  Onee  the  produetion  phase  of  the  installation  is  eomplete  the 
SOVT  is  performed.  A  SOVT  is  performed  by  government  personnel  and  serves  as  the 
aeeeptanee  testing.  Sinee  SSC  LANT  had  more  eolleetive  experienee  with  CSRR  they 
provided  the  majority  of  the  SOVT  SMEs.  The  produetion  agent  provides  support  to  the 
installation  teams  if  a  pieee  of  material  or  equipment  fails  or  is  defeetive.  Onee  SOVT  is 
eomplete  the  platform  assumes  responsibility  for  CSRR  and  all  of  the  aneillary  systems. 

E,  COMMON  SUBMARINE  RADIO  ROOM  SYSTEM  OF  SYSTEMS 

SUSTAINMENT  APPROACH 

Even  though  CSRR  is  a  SOS  there  are  sustainment  responsibilities  to  be 
maintained.  SPAWAR  Systems  Center  Atlantie  is  assigned  as  the  CSRR  in  serviee 
engineering  aetivity  (ISEA)  responsibilities.  The  ISEA  shown  in  Eigure  30  is  responsible 
for  providing  onsite  and  distanee  teehnieal  support,  provides  the  initial  spares  outfitting, 
and  is  the  CSRR  inventory  eontrol  point  for  repair  parts.  The  ISEA  maintains  a  eadre  of 
onsite  representatives  (OSR)  at  most  submarine  ports  whieh  provide  loeal  support  and 
perform  minor  modernization.  The  produetion  agent  is  responsible  for  overseeing  the 
aetivities  of  the  ISEA  and  was  interviewed  as  part  of  the  researeh.  Notes  from  the 
integrated  logisties  support  management  team,  fleet  support  team  and  program 
management  review  aetion  items  were  reviewed  as  part  of  this  researeh.  Obsoleseenee 
management  is  a  reeurring  item  whieh  must  be  managed  by  the  ISEA  working  in  eoneert 
with  the  support  aetivities  of  the  other  PORs. 
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Figure  30.  SPAWAR  Systems  Center  Atlantie  In  Serviee  Engineering  Aetivity 

Organization 


Unlike  Trident  IRR,  whieh  saw  little  change  until  the  introduction  of  submarine 
IP,  both  SCSS  and  CSRR  went  through  a  significant  amount  of  change  as  systems 
became  more  tightly  integrated  and  automated.  The  SCSS  represented  a  transition  from 
manual  patch  panels  to  the  automated  baseband  switching  eliminating  the  manual 
patching  of  crypto  devices  to  other  baseband  equipment  and  consolidating  a  number  of 
individual  systems  into  a  consolidated  modernization  availability.  Common  Submarine 
Radio  Room  introduced  an  integrated  yet  open  architecture  which  automated  RF 
switching  and  expanded  the  network  to  control  the  systems  and  distribute  information  to 
the  appropriate  users. 

Introduction  of  new  changes  normally  results  in  an  increase  in  requests  for 
technical  assistance  until  enough  native  user  knowledge  is  available  to  operate  and 
maintain  their  systems.  One  drawback  of  installing  a  new  system  is  the  lack  of  technical 
expertise  of  how  the  system  interfaces  with  other  systems.  The  OSRs  are  responsible  for 
accomplishing  minor  modernization  and  providing  onsite  technical  assistance  to  develop 
the  core  of  knowledge  so  users  can  determine  if  a  problem  is  the  new  system  or 
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elsewhere.  The  OSR  ean  assess  a  problem  to  determine  if  it  is  CSRR  related  or  eaused  by 
an  individual  system  and  if  neeessary  engage  the  speeifie  POR  SMEs  to  resolve  the  issue. 


F.  COMMON  SUBMARINE  RADIO  ROOM  SYSTEM  OF  SYSTEMS 

TRAINING  APPROACH 

SSC  LANT  is  responsible  for  developing  the  training  material  used  by  Submarine 
Learning  Center  on  their  multi  reeonfigurable  training  system  (MRTS).  The  MRTS  is  a 
eomplete  network  based  training  system  whieh  provides  a  virtual  representation  of  the 
CSRR  using  toueh  sereen  teehnology  mounted  in  full  sized  raeks  representing  a  CSRR 
system.  Prior  to  MRTS  CSRR  training  was  provided  using  training  teehnieal  equipment 
(TTE)  whieh  is  a  physieal  and  funetional  representative  ship  system  installed  in  a  training 
laboratory.  The  advantage  of  this  obviously  is  the  ability  of  the  operator  to  toueh  and 
manipulate  the  systems  just  as  they  would  on  the  platform.  The  disadvantage  of  using 
TTE  is 

1 .  If  the  TTE  breaks  down,  training  value  ean  be  lost. 

2.  In  order  to  replieate  this  eapability,  it  must  be  installed  at  eaeh  site  or 
operators  must  travel  to  the  site  for  training 

3.  There  is  a  signilieant  eost  to  install  and  maintain  TTE.  This  ineludes  a  eost 
on  the  program  to  proeure  additional  assets  inereasing  their  TOC. 

4.  Pre  faulted  modules  and  seenarios  had  to  be  developed.  Oeeasionally 
introdueing  a  fault  eould  aetually  eause  a  real  failure. 

Notes  from  the  submarine  eommunieations  and  networks  training  management 
team  (SCANTMT)  were  reviewed  as  part  of  the  researeh.  A  signifieant  ehallenge  whieh 
eonfronts  the  training  eommunity  eoneerns  the  delivery  of  training  for  a  system  of 
systems.  The  CSRR  training  uses  a  system  of  systems  approaeh  for  operations  and 
maintenanee  while  eaeh  individual  program  provides  their  own  training  solution  and  ILS 
doeumentation.  The  submarine  foree  has  reeognized  the  value  of  using  a  trainer  like 
MRTS  and  is  implementing  similar  approaehes  for  other  system  trainers  ineluding 
weapons  eontrol  and  eleetronie  warfare.  Unfortunately,  the  laek  of  a  eonerete  eommon 
requirement  within  the  system  training  plans  has  resulted  in  training  solutions  whieh  do 
not  synehronize  well. 
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The  SSC  LANT  training  team  is  partnered  with  the  NUWC  ILS  team  whieh  uses 
the  CSRR  doeumentation  and  information  from  the  individual  programs  to  incorporate 
their  systems  information  into  the  MRTS.  There  are  several  advantages  of  using  a  SOS 
solution  like  MRTS 

1 .  While  a  loss  of  a  MRTS  at  a  school  site  can  impact  training,  the  chance  of 
entirely  stopping  training  is  substantially  reduced  since  each  training  site 
has  several  systems  installed. 

2.  The  ability  to  replicate  the  training  capability  has  a  non-recurring  cost  to 
install  the  hardware  and  software.  Once  this  is  complete,  updates  will  be 
developed  by  the  MRTS  team  and  distributed  to  all  sites. 

3.  Removing  the  TTE  eliminates  the  cost  of  installing  a  new  trainer  and 
associated  maintenance  and  modernization  costs. 

4.  The  ability  to  reconfigure  the  trainer  from  one  CSRR  version  to  another 
can  be  accomplished  in  very  little  time. 

5.  The  MRTS  allows  an  operator  to  still  manipulate  his  CSRR  and  get  the 
applicable  responses.  This  includes  training  in  various  mission  scenarios 
and  equipment  casualties. 

The  CSRR  shares  several  characteristics  with  the  Air  Force  KC-135  FTS.  The 
KC-135  FTS  uses  a  hardware  TTF  approach  to  train  flight  crews  (Chislaghi,  Dyer  and 
Free  2010).  Their  approach  shared  the  same  challenges  as  the  TTF  installed  at  the 
training  facilities  in  Bangor,  Washington  and  Kings  Bay,  Georgia.  The  Air  Force 
discovered  like  CSRR  that  resources  had  to  be  allocated  in  order  to  maintain  and  upgrade 
the  trainers.  The  approach  at  this  point  treated  the  trainers  like  platforms  for 
modernization  purposes.  This  approach  was  feasible  as  long  as  changes  were  minimized. 
However,  the  rapid  modernization  occurring  across  the  fleet  forced  acknowledgement 
this  approach  was  no  longer  cost  effective  nor  responsive  enough  to  meet  the  fleet 
operator  needs.  The  transition  to  MRTS  demonstrated  a  SOS  training  approach  could  be 
effectively  developed.  The  success  of  the  MRTS  program  resulted  in  the  submarine  force 
expanding  its  use  to  other  systems. 

G.  CSRR  SOFTWARE  DEVELOPMENT  AND  SUSTAINMENT 

SSC  PAC  is  responsible  for  the  development  and  sustainment  of  the  control  and 
management  (C&M)  software.  The  software  project  manager  works  with  the  prime 
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vendor  to  develop  new  updates  for  eaeh  CSRR  version  via  delivery  orders.  All  other 
programs  provide  their  own  software  to  support  their  systems  but  also  provide 
information  for  enabling  remote  operation  of  their  systems  from  the  C&M.  The  C&M 
software  provides  the  overall  systems  control  and  management  of  CSRR.  The  integration 
of  new  capabilities  identified  from  information  derived  from  the  constituent  programs  to 
create  the  necessary  drivers.  The  advantage  of  this  approach  means  the  individual 
programs  maintain  their  own  software  and  the  C&M  provides  the  overall  capability  of 
tying  the  individual  components  to  each  other,  in  essence  is  the  glue  needed  to  make 
everything  work  efficiently. 

The  challenge  of  managing  the  C&M  software  is  similar  to  other  programs. 
Changing  technology,  new  interfaces,  and  increasing  security  upgrades  via  software 
while  leveraging  deployed  applications  is  a  challenge  for  a  single  program.  This 
challenge  is  no  less  for  the  C&M  which  must  handle  a  multitude  of  components  for 
configuration,  circuit  management  and  system  status. 

H,  CHALLENGES  FACING  COMMON  SUBMARINE  RADIO  ROOM  AS  A 

SYSTEM  OF  SYSTEMS 

The  challenges  facing  CSRR  or  any  other  SOS  share  many  characteristics.  As  an 
acknowledged  SOS  the  CSRR  program  manager  has  the  same  responsibilities  as  his  peers 
managing  their  product  programs.  The  SOS  program  manager  faces  additional  challenges 
to  ensure  the  specific  program  activities  under  his  responsibility  take  into  account  not 
only  his  program  but  the  challenges  and  opportunities  of  his  peers.  If  a  program  attempts 
seeks  to  optimize  their  constituent  system  without  consideration  of  the  impact  it  may 
have  on  others  the  results  may  be  a  more  fragile  system,  vulnerable  to  intentional  or 
unintentional  degradation. 

1.  Program  and  Other  POR  Requirements 

One  of  the  main  challenges  facing  CSRR  is  the  relationship  it  has  with  other 
programs.  DOD  largely  acquires  individual  systems  and  integrates  them  versus 
integrating  capabilities  into  systems  of  systems  upfront.  A  notable  exception  was  the  GPS 
program  which  integrated  several  segments  or  components  to  deliver  capability  under  the 
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lead  of  one  serviee.  Sinee  GPS  is  elassified  as  an  ACAT  I  program,  it  was  provided  the 
authority  to  define  the  overall  SOS  arehiteeture  and  the  overall  system  requirements.  GPS 
ean  be  elassified  as  a  direeted  SOS  since  it  was  built  to  address  a  specific  purpose, 
precise  navigation  and  timing.  The  other  services  can  develop  their  systems  to 
accommodate  their  specific  needs  but  must  still  be  interoperable  within  the  established 
GPS  architecture.  The  JCIDS  approach  was  implemented  in  2002  to  address  shortfalls  in 
the  DOD  acquisition  system.  JCIDS  introduced  a  more  defined  process  of  identifying 
capability  gaps  and  solutions.  However,  this  did  not  specifically  address  how  capabilities 
could  be  delivered  via  a  SOS.  DOD  did  recognize  certain  capabilities  required  the 
integration  of  several  systems  to  create  a  SOS.  Most  of  these  were  acknowledged  SOS 
intended  to  work  together  but  the  emphasis  on  the  system  from  a  budgetary  perspective 
shifted  the  focus  off  the  SOS  and  onto  the  system.  CSRR  faces  this  same  challenge. 
While  it  is  an  ACAT  II  program,  resource  officers  considered  it  a  system  like  many 
others.  This  view  often  results  in  creating  breaks  among  the  various  constituent  systems 
and  the  overall  SOS. 

CSRR  can  be  considered  an  acknowledged  SOS  which  has  its  own  recognized 
requirements  and  objectives,  but  each  of  the  constituent  programs  is  independently 
managed,  funded,  and  sustained.  SWFTS  is  under  the  cognizance  of  PEO  SUB  and  is  not 
a  formal  program,  but  a  managed  agreement  which  shares  many  characteristics  of  a 
directed  SOS.  The  challenge  is  CSRR  is  classified  as  an  ACAT  II  program  responsible 
for  delivering  a  defined  capability  like  other  program  when  viewed  from  the  resource 
sponsor  level.  Budget,  contractual  or  technical  changes  affecting  individual  programs 
within  PEO  C4I  or  PEO  SUB  can  impact  the  overall  C4I  capability  and  potentially  force 
significant  design  changes  to  the  CSRR  architecture  (Darlene  Sullivan  2014,  email 
response  to  questionnaire).  Additional  non-technical  issues  can  occur  when  there  is  a 
turnover  of  personnel.  These  new  personnel  sometimes  require  an  introduction  to 
reinforce  the  “value  added”  the  system  of  systems  approach  such  as  CSRR  provides 
(Darlene  Sullivan  2014,  email  response  to  questionnaire). 

A  persistent  challenge  encountered  with  the  extensive  use  of  COTS  or  new 
technology  has  been  the  late  delivery  of  equipment,  or  if  received,  it  has  not  been  fully 
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tested  (Steve  Devin  2014,  email  response  to  questionnaire).  Some  of  this  is  attributable  to 
poor  eommunieation,  teohnieal  or  programmatie  issues  (Darlene  Sullivan  2014,  email 
response  to  questionnaire).  One  example  is  the  joint  taetieal  radio  system  (JTRS)  airborne 
maritime  fixed  station  (AMF),  an  ACAT  1  program.  JTRS-AMF  was  envisioned  to  be  the 
eommon  replaeement  for  the  different  radios  procured  by  each  service.  The  CSRR  V3 
design  planned  using  JTRS-AMF  as  a  cornerstone  component.  The  inability  to  meet 
milestones,  de-scoping  of  key  requirements  and  cost  overruns  ultimately  caused  the 
program  to  be  cancelled.  The  cancellation  forced  the  program  plans  and  schedules  for 
CSRR  and  other  programs  to  be  revised,  and  last  minute  investigation  into  other  solutions 
to  provide  the  capabilities  to  the  warfighter  was  pursued.  Similar  issues  have  been 
encountered  as  other  systems  failed  to  meet  their  schedules  or  had  funding  cut  from  their 
program. 

Testing  of  a  SOS  poses  a  number  of  challenges  to  validate  and  verify  capabilities. 
Testing  of  individual  systems  can  be  performed  using  clearly  defined  criteria  in  a 
controlled  environment.  Even  these  events  are  solely  focused  on  demonstrating  the 
specific  capabilities  inherent  to  the  system.  Testing  and  evaluation  of  a  SOS  is  more 
difficult  since  the  aggregation  of  individual  requirements  can  result  in  a  testing  event 
which  may  be  very  difficult  or  expensive  to  accomplish.  Common  Submarine  Radio 
Room  has  performance  requirements  defined  in  the  CSRR  CPD  and  SUBECS  CRD,  but 
these  must  be  adjudicated  against  the  individual  systems  to  eliminate  conflicts. 

End  to  end  testing  may  also  identify  a  problem  which  was  not  evident  during 
systems  testing.  This  emergence  may  force  unexpected  changes  to  a  specific  system  or  a 
group  of  systems.  An  acknowledged  SOS,  made  up  of  individual  PORs,  must  come  to 
agreement  about  testing  approaches  and  scope.  Each  system  performs  testing  to  meet 
their  particular  key  performance  parameters  and  key  systems  attributes.  Testing  a  system 
rarely  involves  evaluating  full  end  to  end  performance  except  as  part  of  a  formal 
developmental  test  (DT)  or  operational  test  (OT)  event.  The  aspects  of  a  SOS  from 
stakeholder  involvement  to  performance  and  behavior  have  implications  on  the  testing 
and  evaluation  of  a  SOS.  Table  19  lists  the  aspects  of  a  system  and  acknowledged  SOS  to 
identify  implications  for  SOS  testing  and  evaluation  (Dahmann,  et  al.  2010).  The  CSRR 
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DT  and  OT  approach  used  for  each  version  had  to  take  a  macro  level  view  to  demonstrate 
it  could  meet  the  CPD  requirements.  Any  problems  noted  in  individual  systems  had  to  be 
addressed  since  these  reflected  against  the  overall  operational  effectiveness  and 
suitability.  Using  a  systems  approach  would  not  have  identified  many  of  the  issues  during 
DT  and  OT. 

2.  Integrated  Logistics 

Another  challenge  is  the  myriad  of  integrated  logistics  support  (ILS)  issues  which 
arise  from  managing  a  SOS.  Acknowledged  SOS  architectures  require  close  cooperation 
among  the  different  programs  to  ensure  documentation,  repair  parts  and  intermediate  or 
depot  support  in  place  at  the  right  locations  and  formats.  Each  version  performs  a 
reliability  assessment  to  determine  the  appropriate  quantity  of  spare  parts  to  carry.  The 
type  of  reliability  analysis  performed  is  determined  by  the  type  of  platform.  SSBNs 
perform  a  mission  essential  component  (MEC)  analysis  which  assigns  a  numerical  value. 
A  higher  number  represents  a  more  critical  part.  This  reliability  analysis  identifies  which 
repair  parts  must  be  onboard  the  SSBN  to  support  the  strategic  deterrence  mission.  All 
other  platforms  perform  a  readiness  based  sparing  (RBS)  analysis.  Like  the  MEC  an  RBS 
performs  a  similar  function  to  determine  which  repair  parts  should  be  onboard. 

Each  system  performs  their  own  reliability  analysis  which  can  result  in  different 
sparing  levels  when  the  analysis  is  performed  at  the  SOS  level.  System  maturity  can 
impact  the  accuracy  of  the  sparing  analysis.  New  systems  are  analyzed  using  predicted  or 
vendor  provided  data.  Deployed  systems  can  use  actual  failure  data.  Any  disparities 
between  the  systems  reliability  analysis  and  the  SOS  analysis  must  be  resolved, 
especially  if  a  spare  part  is  determined  to  have  a  high  MEC,  or  additional  spare  parts  are 
needed  to  meet  the  results  of  the  overall  SOS.  A  high  MEC  determines  more  repair  parts 
are  required,  which  in  turn  drives  cost.  Conversely,  if  the  systems  within  the  SOS  share 
the  same  components  the  overall  sparing  quantities  may  be  reduced. 
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Table  19.  System  of  Systems  Test  and  Evaluation  Implieations  (from  Dahmann 

et  al.  2010) 


Aspect 

System 

Acknowledged  System  of  Systems 

SOS  T&E  Implications 

Management  &  Oversight 

Stakeholder 

Involvement 

Clearer  set  of 
stakeholders  and 
aligned  objectives 

Stakeholders  at  both  system  level  and 

SOS  levels  (including  the  system  owners), 
with  competing  interests  and  priorities;  in 
some  cases,  the  system  stakeholder  has  no 
vested  interest  in  the  SOS;  all 
stakeholders  may  not  be  recognized. 

Validation  criteria  more 
difficult  to  establish 

Governance 

Aligned  PM  and 
funding 

Added  levels  of  complexity  due  to 
management  and  funding  for  both  the 

SOS  and  individual  systems;  SOS  does 
not  have  authority  over  all  the  systems. 

Can  not  explicitly  impose 
SOS  conditions  on  system 
T&E 

Operational  Environment 

Operational 

Focns 

Designed  and 
developed  to  meet 
operational 
objectives 

Called  upon  to  meet  a  set  of  operational 
objectives  using  systems  whose  objectives 
may  or  may  not  align  with  the  SOS 
objectives. 

System  level  operational 
objectives  may  not  have 
clear  analog  in  SOS 
conditions  that  need  T&E 

Implementation 

Acqnisition 

Aligned  to  AC  AT 
Milestones, 
documented 
requirements,  SE 

Added  complexity  due  to  multiple  system 
lifecycles  across  acquisition  programs, 
involving  legacy  systems,  systems  under 
development,  new  developments,  and 
technology  insertion;  Typically  have 
stated  capability  objectives  upfront  which 
may  need  to  be  translated  into  formal 
requirements. 

Depends  on  testing  of 
constituent  systems  to 

SOS  requirements  as  well 
as  SOS  level  testing 

Test  & 

Evaluation 

Test  and  evaluation 
of  the  system  is 
generally  possible 

Testing  is  more  challenging  due  to  the 
difficulty  of  synchronizing  across 
multiple  systems"  life  cycles;  given  the 
complexity  of  all  the  moving  parts  and 
potential  for  unintended  consequences 

Difficult  to  bring  multiple 
systems  together  for  T&E 
in  synchrony  with 
capability  evolution. 

Engineering  &  Design  Considerations 

Boundaries  and 
Interfaces 

Focuses  on 
boundaries  and 
interfaces  for  the 
single  system 

Focus  on  identifying  the  systems  that 
contribute  to  the  SOS  objectives  and 
enabling  the  flow  of  data,  control  and 
functionality  across  the  SOS  while 
balancing  needs  of  the  systems. 

Additional  test  points 
needed  to  confirm 
behavior 

Performance  & 
Behavior 

Performance  of  the 
system  to  meet 
specified  objectives 

Performance  across  the  SOS  that  satisfies 
SOS  user  capability  needs  while 
balancing  needs  of  the  systems 

Increased  subjectivity  in 
assessing  behavior,  given 
challenges  of  system 
alignment. 

Legaey  or  non-COTS  systems,  which  may  have  spare  parts  in  short  supply,  add 
new  dimensions  as  well.  The  challenge  to  building  or  modernizing  a  SOS  with  a  legacy 
system  may  determine  additional  repair  parts  are  needed,  only  to  find  out  there  are  no 
spare  parts  available,  or  the  reengineering  costs  exceeds  the  available  resources. 
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Documentation  must  remain  current  as  well  for  eaeh  platforms  configuration. 
This  is  one  area  where  there  is  significant  weakness.  Individual  systems  will  update  their 
operating  proeedures,  maintenanee  manuals  and  software  user  manuals  as  ehanges  oeeur. 
The  format  of  these  manuals  may  vary  as  well  unless  the  standards  are  ineluded  as  a  data 
item  in  the  contract.  Few  SOS  create  overarching  manuals  which  aggregate  the 
information  needed  to  create  consolidated  operating  proeedures.  The  IRR  developed 
integrated  proeedures  and  manuals,  providing  a  standard  approaeh  for  CSRR  whieh  is 
used  today.  Overarehing  technical  manuals,  such  as  cable  manuals,  are  created  as 
referenees  to  support  maintenance  and  repairs.  The  quality  of  overarching  documentation 
is  directly  related  to  the  quality  of  the  souree  data.  If  the  data  is  poor,  extra  work  may  be 
required  to  improve  the  quality. 

3,  Training 

System  of  systems  training  is  a  relatively  new  eoneept  for  DOD.  The  approaeh 
used  by  CSRR  is  a  large  step  in  the  right  direetion  but  the  solution  is  imperfeet.  Effective 
systems  engineering  considers  every  facet  to  ensure  they  develop  an  appropriate  solution. 
The  SOS  engineer  must  consider  how  the  training  solution  impacts  the  desired  end  state. 
If  there  is  little  need  to  an  operator  to  interfaee  extensively  with  the  system  or  there  is  a 
large  cadre  of  onsite  teehnicians  the  training  solution  may  be  minimal.  If  there  is  a  need 
to  train  operators  to  respond  to  a  complex  scenario  involving  extensive  C4I  capabilities 
sueh  as  earner  strike  group  performing  strike  operations  in  concert  with  a  cyber-operation 
or  humanitarian  aid  /  disaster  relief  the  current  trainers  cannot  be  networked  to  support 
this  and  eoordinating  operational  assets  is  time  eonsuming  and  eostly.  A  SOS  engineer 
has  the  task  of  examining  the  SOS  arehiteeture  to  come  up  with  a  balaneed  approaeh  to 
the  solution. 

To  date  there  is  little  to  no  policy  or  guidance  for  managing  the  training  of 
multiple  systems  integrated  together.  Eaeh  system  is  required  to  develop  a  training 
systems  plan  whieh  is  typieally  not  eoordinated  with  other  systems.  This  in  turn  results  in 
training  materials  and  eurrieula  to  operate  and  maintain  the  speeific  AN/USQ-XX  but 
little  is  eovered  about  how  it  fits  into  the  larger  SOS  arehiteeture.  A  training  eourse  for  a 
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technician  may  train  them  on  a  variety  of  equipment  but  there  is  very  little  about  how 
they  are  related  at  a  larger  level.  One  limitation  to  this  is  the  cost  to  build  a  SOS  trainer  is 
very  cost  prohibitive  and  the  student  throughput  is  limited  to  a  predetermined  number  for 
each  course.  An  option  to  meet  this  need  is  to  build  a  virtual  type  trainer. 

To  meet  this  need  the  CSRR  program  developed  a  Multi-Reconfigurable  Training 
System  (MRTS).  The  MRTS  is  a  virtual  representation  of  the  CSRR  which  is  used  to 
train  operators  and  maintenance  technicians.  The  system  is  composed  of  touch  screen 
monitors  mounted  in  the  racks  similar  to  the  platform  and  arranged  to  match  the  platform 
configuration.  The  MRTS  software  emulates  the  real  equipment  and  is  loaded  with  a 
comprehensive  suite  of  scenarios.  Several  advantages  of  this  approach  include; 

•  Lower  costs  to  develop  and  update  the  trainer.  The  initial  startup  costs 
cover  the  hardware  and  initial  software  load.  Installing  a  complete  suite  of 
technical  training  equipment  (TTE)  can  cost  $20  million  and  about  $500 
thousand  annually  for  maintenance.  Procuring  and  installing  a  MRTS  is 
less  than  $1  million.  Updates  can  be  created  once  and  deployed  to  all  sites. 
Software  development  costs  may  vary  but  are  still  significantly  less  than 
hardware  modernization  and  sustainment. 

•  The  MRTS  can  be  updated  quickly  via  a  software  load.  Hardware  updates 
occur  only  as  needed  to  address  obsolescence  issues.  TTE  remains  in  a 
static  condition  until  it  is  modernized.  Once  TTE  is  modernized  it  cannot 
support  earlier  configurations  for  training. 

•  If  necessary  a  MRTS  lab  can  be  reconfigured  to  a  different  version  or 
version  represented  on  another  platform.  Other  PORs  such  as  ADNS, 
SUBLAN,  CANES  and  team  trainers  can  leverage  the  MRTS  approach.  A 
TTE  laboratory  is  limited  to  the  installed  configuration. 

Increasing  information  assurance  and  cybersecurity  requirements  consume  a 
larger  role  of  training  technicians  and  operators.  The  approach  used  today  by  the 
submarine  force  to  address  communications  and  networks  is  under  the  responsibility  of 
two  different  ratings.  Communications  systems  operations  and  maintenance  is  the 
responsibility  of  the  submarine  communications  electronics  technicians  (ETR).  All 
network  responsibilities  are  managed  by  the  submarine  information  systems  technician 
(ITS).  Current  DOD  policy  mandates  all  personnel  assigned  duties  to  work  on  a  network 
must  be  a  certified  member  of  the  information  assurance  workforce  (lAWF).  This 
mandate  did  not  discriminate  between  closed  and  open  networks.  The  mandate  created  a 
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substantial  amount  of  confusion  for  the  technical  ratings  in  terms  of  where  the  line  of 
separation  is  defined  between  isolated  networks  used  for  the  control  of  systems  and  those 
responsible  for  managing  the  flow  of  data  from  one  point  to  another. 

The  duties  of  many  of  these  technical  rates  mean  they  must  have  a  substantial 
level  of  access  to  operate  their  systems.  Specifically  a  problem  arises  if  a 
communications  component  in  the  control  network  fails  the  ITS  must  be  notified  in  order 
to  investigate  and  correct  the  problem.  The  challenge  is  the  ITS  has  not  received  any 
training  on  the  CSRR  as  a  SOS  so  they  must  rely  on  their  basic  network  knowledge.  The 
training  provided  to  the  ETR  does  not  include  any  network  systems  which  limits  them  to 
identifying  which  network  component  might  have  a  problem.  The  conflict  which 
frequently  arises  from  this  dichotomy  is  if  the  problem  has  several  potential  causes  a 
great  deal  of  back  and  forth  exchange  occurs  in  order  to  fully  understand  if  the  failure  is 
truly  a  network  problem  or  a  communications  component  interfaced  to  the  network.  The 
lack  of  providing  adequate  training  to  both  ratings  creates  a  gap  in  their  knowledge  which 
creates  a  risk  of  a  platform  incurring  a  communications  outage. 

The  solution  to  this  would  be  to  (1)  designate  the  ETR  personnel  as  member  of 
the  lAWE  or  (2)  initiate  a  rating  conversion  of  all  ETR  personnel  to  ITS  which 
automatically  places  them  in  the  lAWE,  or  (3)  create  a  designation  criteria  of  which 
systems  require  an  lAWE  certified  technician  and  which  ones  can  be  maintained  by  other 
personnel.  Within  the  submarine  force  this  issue  is  not  isolated  to  the  ETR  rating.  Other 
ratings  including  combat  systems  and  engineering  ratings.  On  a  macro  level  this  problem 
is  not  isolated  to  the  submarine  force.  The  Navy  as  a  whole  faces  challenges  to  determine 
how  to  fully  train  and  equip  their  forces  when  many  systems  are  now  designed  to 
interoperate  so  closely. 

4,  Production 

The  versions  of  CSRR  prior  to  V3  were  managed  solely  by  SSC  LANT.  During 
this  period  the  production  team  and  installation  teams  worked  closely  together  to  build 
each  CSRR  prior  to  shipping  to  the  installation  site.  The  number  of  platforms  in  each 
class  also  helped  since  they  were  small  enough  to  create  a  single  design  which  could  be 
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fielded  in  eoneert  with  the  modernization  periods.  After  assuming  responsibility  for  the 
CSRR  program  and  as  lead  systems  integrator  PMW770  developed  and  installed  CSRR 
on  the  SSGNs  replaeing  their  legacy  Trident  IRRs  using  the  VA  CSRR  design  as  a 
model.  The  SSBNs  followed  closely  leveraging  the  work  from  the  SSGN  design  while 
incorporating  the  components  and  systems  necessary  to  support  their  strategic  deterrence 
mission.  The  Seawolf  class  followed  next  leaving  the  LA  class  as  the  final  exception.  The 
production  teams  would  build  each  ship  set,  perform  a  PITCO,  configure  it  for  the 
designated  platform,  an  then  pack  and  ship  it. 

The  deployment  of  V3  coincided  with  the  standup  of  the  FRD  and 
implementation  of  the  global  installation  contract.  Prior  to  V3  the  production  team  would 
coordinate  the  delivery  of  all  components  and  systems  after  performing  a  complete 
PITCO  of  the  ship  set.  V3  proved  to  be  a  coordination  challenge  much  more  significant 
than  earlier  versions.  First,  the  initial  platform  to  receive  V3  was  the  LA  class,  which  also 
had  the  largest  population  of  platforms  and  all  of  them  required  major  modernization 
with  a  total  replacement  of  their  SCSS.  Second,  the  number  of  new  systems  making  up 
V3  each  had  their  own  mounting  kits,  software,  alteration  documentation,  testing 
requirements  and  sparing  approaches.  The  CSRR  V3  production  team  had  to  contend 
with  these  issues  as  well  as  the  differing  bill  of  materials  developed  for  each  class.  Third, 
some  PORs  insisted  in  delivering  their  systems  directly  to  the  installation  site  vice  having 
it  go  through  the  PITCO  process.  This  increased  the  risks  of  finding  infant  failures  which 
delayed  testing  and  drove  up  costs. 

Several  lessons  were  learned  from  the  production  process  for  V3.  First,  the 
production  team  manufactured  all  of  the  cables  for  the  installation  with  at  least  one  end 
completed  in  order  to  accelerate  the  installation.  Second,  the  design  drawings  provided  by 
the  design  agent  were  used  to  pre-assemble  pieces  into  larger  subassemblies  again  to 
accelerate  the  installation.  A  problem  which  arose  from  this  approach  is  the  AIT  received 
the  same  drawings  and  would  report  materials  missing  which  were  actually  consumed  in 
the  pre-fabrication  process.  Third,  the  production  team  developed  the  integrated  SOVT 
merging  all  of  the  system  verification  testing  into  a  larger,  more  comprehensive  systems 
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verification.  A  result  of  this  discovered  the  SPAWAR  system  for  tracking  installation 
completions  does  not  have  the  capability  of  tracking  the  status  of  SOS  installation  and 
testing  progress. 

5.  Installations  and  Synchronization  of  Installations  into  Block  Upgrades 

The  establishment  of  the  FRD  significantly  altered  the  existing  installation 
process.  Instead  of  each  program  office  working  directly  with  the  IMO  all  functions  were 
redirected  to  the  FRD  as  the  single  point  of  contact  for  all  installation  issues.  Additionally 
the  FRD  and  IMO  released  a  global  installation  contract  which  supported  multiple  award 
contracts  and  divided  the  installation  responsibilities  geographically  with  SSC  LANT 
responsible  for  the  east  coast  and  SSC  PAC  responsible  for  the  west  coast.  The  standup 
of  the  FRD  shifted  the  installation  responsibilities  away  from  the  CSRR  program 
production  team.  The  shift  significantly  changed  the  relationship  between  the  production 
and  installation  teams  and  eliminated  the  SSC  LANT  installation  learning  curve.  The  new 
installation  contract  discouraged  development  of  a  learning  curve  as  new  vendors  with  no 
prior  CSRR  experience  were  awarded  the  work.  The  lack  of  prior  experience  resulted  in 
the  CSRR  production  team  altering  their  kitting  approach  as  each  new  vendor  wanted  the 
installation  kit  created  and  delivered  differently.  Ultimately  the  FRD,  IMO,  and 
production  team  agreed  on  a  standardized  approach  to  how  the  kits  would  be  produced, 
packed  and  shipped.  However  this  did  not  address  the  issue  of  no  effective  learning  curve 
for  the  AIT. 

PEO  C4I  established  a  strategic  goal  for  reducing  the  number  of  system  variants 
installed  afloat  and  ashore.  To  achieve  this  goal  PEO  C4I  developed  a  synchronized 
fielding  plan  to  align  the  fielding  of  systems  to  platform  availabilities.  This  approach  was 
an  important  first  step  but  overlooked  is  the  system  development  cycle.  In  the  case  of 
CSRR  V3  several  systems  including  ADNS,  NMT,  REDACS  and  new  workstations  were 
consolidated  into  a  single  package.  The  recommended  approach  for  a  development  cycle 
is  working  to  an  event  based  schedule.  However,  this  approach  conflicts  with  the 
calendar  and  budget  based  schedule.  This  is  particularly  acute  when  they  have  not  been 
adjudicated  to  identify  and  mitigate  schedule  conflicts.  The  first  installation  of  CSRR  on 
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an  LA  was  planned  for  early  2012.  In  order  to  meet  the  program  schedule  several 
significant  design  issues  had  to  be  resolved  before  SDVT  and  SAT  could  be 
accomplished.  Solutions  to  these  issues  were  identified  prior  to  the  installation  but 
required  procuring  new  materials  and  revising  the  installation  drawings  in  order  to  allow 
prospective  vendors  to  bid  on  the  work.  Daily  meetings  to  track  the  status  of  material 
occurred  between  the  program  office,  design  agent  and  production  agent.  While  this 
effort  succeeded  in  obtaining  the  materials  and  getting  them  to  the  site  a  key 
consideration  from  a  systems  engineering  and  more  importantly  a  SOS  perspective  is 
attempting  to  identify  and  resolve  issues  as  early  as  possible.  This  may  sound  like 
common  sense  but  a  good  heuristic  is  “Plan  for  the  worst  while  praying  for  the  best.” 

6,  Cybersecurity 

Information  Assurance,  or  more  recently  known  as  Cybersecurity,  has  increased 
in  importance  as  DOD’s  and  the  Navy’s  reliance  on  networks  has  grown.  Prior  to  SCSS 
threats  of  cyber- attacks  were  practically  unknown.  Today  cyber-attacks  and  scans  for 
network  vulnerabilities  occur  constantly.  The  importance  of  protecting  the  components  in 
a  network  such  as  CSRR  or  SWFTS  or  SUBLAN  is  a  clearly  understood  requirement. 
The  challenge  faced  here  is  the  often  arbitrary  approach  to  address  cybersecurity.  By 
arbitrary  it  does  not  mean  policies  and  rules  are  being  ignored.  More  accurately  it  means 
the  application  of  cybersecurity  architecture  approach  at  the  systems  level  often  conflicts 
with  the  architecture  approach  needed  for  a  SOS  due  to  the  interpretation  of  the 
standards.  For  example  applying  security  technical  implementation  guides  (STIG)  to 
individual  systems  is  an  appropriate  approach  if  it  is  meant  to  operate  in  an  isolated  or 
standalone  manner.  Getting  authorization  to  waive  or  modify  cybersecurity  applications 
is  often  complicated  and  time  consuming. 

The  transition  from  the  defense  information  assurance  certification  and 
accreditation  program  (DIACAP)  to  the  defense  information  assurance  risk  management 
framework  (DIARMF)  shown  in  Figure  31  represents  an  opportunity  to  design  and 
implement  more  logical  and  effective  approaches  to  lA  and  network  defense.  The 
DIARMF  places  more  emphasis  on  managing  the  risk  of  a  system  getting  penetrated  vice 
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blindly  following  checklists  which  have  little  regard  for  the  impact  to  the  SOS.  The 
approach  a  system  follows  using  the  DIARMF  is  the  same  approach  the  SOS  lA  engineer 
would  follow. 


DIARMF 


Security  Categorization 
FIPS  199 


Control  Selection 
CNSSI 1253 


Implementation 
NIST  SP  800  53A 


Assessment 

and 

Authorization 

(A&A) 

NIST  SP  800  37 


Continuous  Monitoring 
NISTSP  800-137 


Figure  3 1 .  DIACAP  to  DIARMF  Evolution  (from  Elamb  2013) 
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7. 


Sustainment 


Sustainment  of  CSRR  is  challenging  due  to  the  heavy  reliance  on  COTS  by  all  of 
the  POR  systems  including  those  components  provided  by  the  CSRR  program.  CSRR  is 
not  alone  facing  this  challenge  but  the  frequency  of  changes  occurring  to  the  other 
systems  can  have  a  unforeseen  impact  on  the  overall  SOS  if  not  assessed  and  possibly 
tested.  The  standard  evolution  of  new  technology  is  approximately  18  months.  This  is  far 
faster  than  the  typical  defense  acquisition  program  which  might  not  have  funding  to 
begin  design  work  for  two  years.  Once  funding  is  available  it  is  unlikely  a  design  will  be 
completed  in  18  months.  Implementing  the  design  in  the  current  24  month  development 
period  means  at  least  one  component  that  has  reached  end  of  life.  Performing  a  six  to 
eight  year  modernization  cycle  for  all  platforms  means  the  systems  have  a  large  number 
of  obsolete  components  within  the  first  several  installations. 

SWFTS  encountered  this  issue  with  their  tech  insertion  /  advanced  processor 
build  (TI/APB)  approach.  Similar  to  CSRR  each  TI/APB  delivers  an  integrated  suite  of 
capabilities  for  the  combat  systems.  A  TI/APB  is  released  every  two  years  with  the  TI 
providing  the  hardware  updates  and  the  APB  all  software  updates.  Each  TEAPB  is 
planned  to  modernize  a  certain  number  of  platforms  before  moving  onto  the  next 
upgrade.  The  challenge  is  a  change  of  a  modernization  schedules  creates  the  risk  of 
allowing  a  system  to  exceed  its  planned  obsolescence  and  may  have  difficulties  obtaining 
repair  parts. 

CSRR  faced  the  same  challenge  when  developing  V3.  The  length  of  time  between 
the  initial  design  of  the  first  platform  in  2009  to  completing  the  design  of  the  last  class  in 
2014  resulted  in  a  number  of  obsolescence  issues  which  in  turn  forced  a  number  of 
design  changes.  The  timing  of  the  issue  can  impact  the  ability  to  sustain  the  systems  out 
there  while  introducing  new  challenges.  One  solution  may  be  to  perform  a  lifetime  buy  to 
provide  enough  spares  in  inventory  until  a  new  solution  is  identified  and  deploys. 
Another  is  working  the  users  to  move  the  component  from  one  platform  to  another  to 
support  the  mission  tasking.  An  approach  to  reduce  obsolescence  impacts  may  be 
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introducing  smaller  but  more  frequent  ehanges  viee  over  a  longer  period  with  larger 
changes.  In  either  the  case  the  SOS  engineer  must  still  assess  the  impacts  these  ehanges 
will  have  on  the  overall  SOS  architecture. 

This  problem  is  not  unique  to  CSRR  or  any  of  the  related  systems.  The  ehallenge 
to  the  systems  engineer  is  finding  a  suitable  replacement.  The  challenge  to  SOS  engineers 
is  minimizing  the  impacts  these  changes  induce  in  the  overall  SOS.  The  upgrading  of  a 
loeal  software  application  on  one  system  may  degrade  or  disable  remote  operations 
located  in  another  system.  One  recommendation  to  mitigate  this  is  to  provide  loose 
coupling  within  the  SOS  but  the  extent  to  which  loose  coupling  can  be  applied  varies 
from  system  to  system  within  the  whole  SOS  (Director,  Systems  and  Software 
Engineering  2008,  23). 

I.  RESEARCH  QUESTIONS 

At  this  point  the  following  can  be  summarized  from  the  researeh  into  the 
background  of  CSRR  and  system  of  systems  engineering. 

1.  What  is  Common  Submarine  Radio  Room  and  what  Characteristics 
Classify  it  as  a  System  of  Systems? 

Common  Submarine  Radio  Room  is  an  open  arehitecture  system  of  systems 
developed  to  support  the  submarine  force  communication  requirements.  This  question 
determined  if  CSRR  was  a  SOS  justifying  a  case  study  analysis.  If  so,  then  what 
characteristics  from  the  available  literature  regarding  SOS  would  apply.  The  program 
doeumentation  deseribes  CSRR  as  a  SOS  but  did  not  elaborate  on  specific  characteristics. 
Vaneman  (2013,  13)  applies  a  “litmus  test”  to  determine  the  applicability  of  a  SOS. 
These  consider  if  the  individual  systems  “(1)  Can  operate  independently  from  the  SOS. 
(2)  Have  life-cyeles  that  are  individually  managed.  (3)  Come  together  to  achieve  a 
eapability  that  is  unrealized  by  a  single  system  alone”  (Vaneman  2013,  13).  Using  this  as 
a  litmus  test  for  CSRR  the  following  is  identified  in  answering  the  previous  questions. 
Additional  characteristics  are  listed  in  Table  19. 

(1)  Can  the  individual  systems  operate  independently  from  the  SOS?  Each  of  the 

systems  within  CSRR  is  an  individual  FOR  which  had  to  provide  an  operational  view 
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identifying  their  role  in  the  overall  arehiteeture.  NMT  is  one  example  whieh  has  been 
installed  on  SCSS  platforms  and  ean  operate  independently  of  the  CSRR  SOS. 

(2)  Is  the  life  eyele  of  eaeh  system  individually  managed?  Eaeh  system  sueh  as 
ADNS,  RFDACS  and  NMT  within  CSRR  has  their  own  life  eyele  management. 
Colleetive  assessments  are  done  with  eaeh  version  to  determine  the  SOS  impaets  when 
ehanges  are  made. 

(3)  Do  the  systems  eome  together  to  aehieve  a  eapability  that  is  unrealized  by  the 
single  system  alone?  Eaeh  individual  is  eapable  of  providing  a  eertain  level  of  eapability 
by  itself.  Prior  to  installing  RFDACS  antenna  switehing  functions  were  performed  using 
manual  patch  panels.  Integrating  REDACs  with  the  available  radios  and  antennas 
provides  automation  and  simultaneity  capabilities  that  did  not  exist  previously.  NMT  can 
be  a  standalone  system  but  can  provide  asymmetric  capabilities  when  paired  with  UHE 
and  network  systems. 

Jamshidi  (2009)  identified  the  characteristics  describing  a  SOS  listed  below  in 
Eigure  32  and  Table  20.  Resilience  can  be  further  classified  examining  the  attributes  of 
capacity,  flexibility,  tolerance,  and  cohesion  which  are  supported  by  14  principles 
(Jackson  and  Eerris  2012,  155).  These  characteristics  were  compared  against  CSRR  to 
determine  where  it  compared.  The  type  of  SOS  is  derived  from  the  System  Engineers 
Guide  for  Systems  of  Systems  (Director,  Systems  and  Software  Engineering  2008). 
Governance  is  addressed  using  the  information  provided  from  the  Naval  Postgraduate 
School  System  of  Systems  Engineering  and  Integration  course  (Vaneman,  2013).  These 
criteria  were  used  to  examine  CSRR  and  determine  if  these  characteristics  can  be  used  to 
describe  it  as  a  SOS. 

Governance  is  addressed  in  more  detail  since  this  is  of  particular  interest  to  many 
systems  engineers  and  program  managers.  Governance  is  defined  as  “the  organization, 
set  of  rules,  policies,  and  decision-making  criteria  that  will  guide  a  System  of  Systems 
(SOS)  to  achieving  its  goals  and  objectives”  (Vaneman  2013,  6).  The  characteristics 
listed  in  the  center  of  Eigure  32  and  described  in  Table  21  define  the  two  extremes  of 
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defining  a  SOS.  Governance  of  a  SOS  has  four  criteria  defined  in  Table  21  which  must 
be  considered  in  order  to  determine  if  there  is  effective  management. 


Figure  32.  System  of  Systems  Characteristics  Spectrum  (after  Vaneman  2013, 

20) 
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Table  20.  System  of  Systems  Characteristics  and  Applicability  to  Common 
Submarine  Radio  Room  (after  Jamshidi  2009) 


SOS 

Characteristics 

Description 

Applicability  to  CSRR 

Type  of  SOS 
(Director, 

Systems  and 
Software 
Engineering, 
2008) 

Ad  hoc,  virtual,  acknowledged  and 
directed 

CSRR  is  considered  an 
acknowledged  SOS  since 
there  is  an  assigned 
manager,  funding  and 
resources  but  must 
collaborate  with  other 
programs  in  order  to  deliver 
full  capability 

Evolutionary 
behavior 
(Jamshidi  2009, 
193-194) 

The  evolution  of  a  SOS  can  select  or 
eliminate  system  configurations 
independently  of  the  presence  of  other 
configurations”  as  long  as  the 
configurations  are  not  subsequent 
system  states 

CSRR  has  demonstrated 
evolutionary  characteristics 
as  new  technology  is 
incorporated.  Each  change 
to  add  or  remove  a  system 
has  been  assessed  to 
determine  its  impact  to  the 
required  capabilities 

Self- 

Organization 
(Jamshidi  2009, 
194) 

Operational  independence  signifies  that 
subsystems  of  an  SOS  are  independent 
and  useful  in  their  own  right. 

Managerial  independence 
signifies  that  a  system  both  is  able  to 
operate  independently  and  actually  is 
operating  independently. 

Operational  independence  is 
reflective  of  the  systems 
comprising  CSRR.  They  are 
independent  from  a  funding 
and  managerial  perspective. 

Heterogeneity 
(Jamshidi  2009, 
194) 

Heterogeneity  is  a  strong  driver  of 
system  complexity.  A  system  is  often 
heterogeneous  on  multiple  layers 
simultaneously  (e.g.,  size,  architecture, 
life  cycle,  scientific  area,  and 
elementary  dynamics) 

CSRR  can  be  considered 
heterogeneous  since  it  is 
composed  of  a  variety  of 
systems  to  provide 
capability  to  the  user 

Emergence  or 
Emergent 
Behavior 
(Jamshidi  2009, 
85-86,  194; 
Vaneman  2013, 
47) 

The  unexpected  appearance  of  new 
properties  in  the  course  of  development, 
evaluation,  and  operations 

Two  types:  weak  and  strong. 

Weak  emergence  can  be  foreseen 
through  experience  or  modeling  and 
simulation. 

Strong  emergence  is  indeterminate 

Weak  emergence  is 
addressed  through  the 

CSRR  development 
approach  and  experiences 
from  the  deployment  and 
sustainment 

Redundancy 
(Jamshidi  2009, 
199) 

Traditional  SOS  are  often  designed 
with  multiple  redundant  high-level 
subsystems;  i.e.,  a  functional 
requirement  is  satisfied  by  multiple 

CSRR  demonstrates  Type  I 
qualities  through  the  use  of 
multiple  communications 
paths  and  networks 
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SOS 

Characteristics 

Description 

Applieability  to  CSRR 

design  parameters. 

Type  1  have  redundancy 

Type  11  has  no  redundancy. 

Autonomy 
(Jamshidi  2009, 
48;  Vaneman 
2013,47) 

The  ability  to  make  independent 
choices  or  conform  to  a  higher  standard 
Two  key  aspects  of  system  autonomy 
that  must  be  preserved;  technical 
autonomy  and  operational  autonomy. 

Each  of  the  systems 
comprising  CSRR  have 
their  own  teehnical  and 
programmatic  requirements 
and  can  be  considered 

autonomous. 

Diversity 
(Jamshidi  2009, 
49) 

Diversity  of  needs  and  environmental 
diversity 

Can  be  Homogeneous  or 

Heterogeneous 

Each  system  in  CSRR 
provides  a  capability  but 
there  is  a  large  amount  of 
commonality  in  the  design 

Complexity 
(Jamshidi  2009, 
45) 

The  use  of  existing  systems  to  create 

SOS  solutions  introduees  unavoidable 
eomplexities,  both  in  terms  of 
constraints  and  consequences 

CSRR  has  to  deal  with  a 
number  of  technical, 
programmatic  and  funding 
constraints 

Net  Centrieity 
or  Connectivity 
(Director, 

Systems  and 

Software 

Engineering 

2008,  9; 

Vaneman  2013, 
47) 

Net  centrieity  is  relevant  to  all  SOS 
within  DOD 

Is  the  eonnectivity  more  platform 
centric  or  network  eentric 

CSRR  was  designed  to 
support  net  centric 
operations  from  the 
submarine  platform 

Belonging 
(Vaneman  2013, 
47) 

To  be  a  member  of  a  group  or  to  have 
qualifications 

This  can  be  eentralized  or  decentralized 

An  acknowledge  system 
can  choose  whieh  systems 
to  include  or  not.  CSRR  has 
a  formal  relationship  with 
other  PORs  to  deliver 
capability 

Connectivity 

(Vaneman 

2013,47) 

Is  eonnectivity  more  platform  centric  or 
network  centric 

CSRR  is  designed  to 
support  the  submarine  foree 
but  the  principles  could  be 
applied  to  other  platforms 

Resilience 
(Jackson  and 
Ferris  2012, 

153) 

The  ability  to  adapt  to  changing 
conditions  and  prepare  for,  withstand, 
and  rapidly  recover  from  disruption 

CSRR  as  a  SOS 
incorporates  a  design 
capable  of  preparing  for 
disruptions  and  if  disrupted 
can  recover  within  the 
required  eriteria 

Governance 

The  organization,  set  of  rules,  policies 

Governance  is  managed 
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SOS 

Characteristics 

Description 

Applicability  to  CSRR 

(Vaneman  and 
Jaskot  2013) 

and  decision  making  criteria  that  guide 
a  SOS  to  achieve  its  goals  and 
objectives 

through  program  policy  and 
guidance. 

Table  2 1 .  Governance  Criteria  (from  Vaneman  20 1 3) 


Criteria 

Description 

Application 

Criteria  1: 

Organizational 

Structure, 

Standards  and 
Policies 

The  organizational 
structure,  standards, 
policies,  and 
management 
environment  must  be 
understood  to  develop 
effective  governance. 

To  be  successful,  governance  must  be 
consistent  with  the  organization 

Criteria  2: 
Governance 
Composition  and 
Principles 

Determines  the  degree  of 
participation, 
responsiveness, 
consensus,  inclusiveness, 
and  accountability 
needed  in  the  governance 
strategy 

Virtual-  SOS  participants  not  included  in 
the  decisions  of  suggested  changes. 
Directed-  High  degree  of  participation, 
inclusiveness,  responsiveness,  and 
consensus. 

Criteria  3: 
Encapsulation 

How  transparent  are  the 
governance  decisions, 
and  how  is  enforcement 
managed  within  the  SOS 

Virtual-  Most  stakeholders  do  not  care 
as  long  as  they  can  achieve  their 
missions  and  goals. 

Directed-  Governance  strategy  is 
required  to  be  more  inclusive  and 
transparent. 

Criteria  4; 
Governance 
Effectiveness  and 
Interoperability 

Determines  the 
effectiveness  and 
interoperability  attributes 
of  the  SOS 

Virtual-  Used  for  their  own  purposes. 
Should  favor  independence  and 
decentralization.  Difficult  to  predict  or 
measure  effectiveness. 

Directed-  Designed  to  work  together 
toward  a  common  objective. 

Effectiveness  and  interoperability  should 
focus  on  engineered  effectiveness 
standards  and  tightly  controlled  interface 
standards. 
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Based  on  the  eharaeteristies,  CSRR  ean  be  elassified  as  an  aeknowledged  SOS. 
An  aeknowledged  SOS  has  reeognized  objeetives,  a  designated  manager,  and  resourees 
assigned.  The  individual  systems  are  managed  separately  in  terms  of  ownership,  funding, 
and  sustainment.  System  ehanges  are  primarily  managed  by  the  parent  program  but  are 
elosely  eollaborated  with  the  CSRR  program.  Many  programs  within  DOD  ean  be 
eonsidered  acknowledged  SOS  since  they  started  out  as  a  standalone  or  stovepipe  system 
and  over  time  were  integrated  to  create  or  deliver  capabilities  needed  by  the  user. 

2.  What  are  the  Benefits  and  Challenges  of  Developing,  Designing, 
Producing,  Deploying,  and  Sustaining  Common  Submarine  Radio 
Room  as  a  System  of  Systems? 

Prior  to  CSRR  the  typical  approach  to  deliver  a  capability  occurred  in  stovepipe 
fashion.  Stovepipe  systems  provide  all  of  the  components  within  the  overall  program. 
The  disadvantage  of  this  approach  is  costs  can  be  prohibitive,  logistics  can  be  very 
complex  and  prone  to  proprietary  issues  and  performance  may  be  limited  to  a  very  small 
set  of  requirements.  For  a  submarine  space  and  weight  are  critical  considerations  when 
determining  what  systems  are  needed.  In  the  late  1990s  the  emphasis  on  COTS  drove 
many  programs  to  provide  their  own  controller  for  their  system,  normally  in  the  form  of  a 
laptop  or  desktop  workstation.  The  space  limitations  in  the  submarine  radio  room 
immediately  identified  a  need  for  establishing  some  means  of  centrally  controlling  all  of 
the  systems.  The  Trident  IRR  provided  a  central  control  capability  but  used  a  proprietary 
architecture.  Additionally  the  following  comment  provides  more  insight  into  IRR 

IRR  was  actually  the  first  real  implementation  of  an  integrated  submarine 
communications  system,  although  not  really  a  SOS.  While  well-designed 
with  excellent  reliability,  necessary  modernization  became  cost 
prohibitive.  In  the  days  of  shrinking  budgets,  dedicated  interface  protocols 
and  bus-based  systems  have  become  unaffordable  (Darlene  Sullivan  2014, 
response  to  questionnaire) 

As  SCSS  was  deployed  in  response  to  the  IMCS  MNS,  it  served  as  the  precursor 
for  CSRR.  The  following  quote  captures  some  of  the  differences  between  SCSS  and 
CSRR. 
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SCSS  was  the  first  government  integrated  submarine  eommunieation 
system.  The  most  signifieant  arehiteetural  differenee  between  SCSS  and 
CSRR  is  the  use  of  baseband  switching.  The  miniaturization  of  crypto  and 
the  implementation  of  network  interfaces  obviated  the  need  for  baseband 
switching  and  removed  throughput  limitations.  Additionally,  the 
Integrated  Network  Manager  (INM)  in  the  SCSS  controlled  primarily  the 
baseband  switch  (not  the  whole  room),  and  changes  to  the  baseband 
switch  could  be  made  easily,  with  updates  to  a  database  vs  the  INM 
software.  However,  these  two  technical  differences  both  also  have 
programmatic  implications  for  modernization.  The  baseband  switch 
allowed  for  a  more  well  defined  boundary  and  easier  division  of 
responsibility  between  PORs  during  the  modernization  phase. 
Additionally,  software  configuration  updates  for  SCSS  were  mostly  done 
with  database  changes;  minimal  software  development  was  required. 
(Darlene  Sullivan  2014  response  to  questionnaire) 

The  SCSS  enabled  the  operator  to  be  more  effective  but  the  multitude  of 
individual  controllers  degraded  their  ability  to  maintain  situational  awareness  of  the 
communications  status.  Delivering  individual  systems  also  created  problems  for 
configuration  management  and  ITS.  Since  systems  were  delivered  individually  the 
combination  of  different  configuration  increased  exponentially.  Just  modernizing  four 
systems  could  result  in  potentially  16  different  configurations  to  track.  Of  the  42  LA 
submarines  there  were  essentially  42  different  configurations.  The  changes  may  have 
been  minor,  but  the  lack  of  accurate  ILS  documentation,  training,  and  technical  support 
made  design  changes  and  sustainment  more  challenging. 

3,  What  Best  Practices  have  heen  Identified  and  Implemented  in  the 
Common  Submarine  Radio  Room  Program  and  what  Benefits  have 
heen  realized  in  Terms  of  Cost,  Performance,  and  Schedule? 

The  Department  of  Defense  has  implemented  a  number  of  initiatives  to  improve 
efficiency,  contain  costs  and  deliver  capability.  A  major  goal  of  these  initiatives  included 
identifying  effective  means  of  measuring  program  success  and  progress.  The  ability  to 
measure  progress  using  lagging  indicators  is  relatively  easy  to  do  since  elapsed  time  and 
performance  criteria  can  be  measured.  The  drawback  of  this  approach  is  changes  to  a 
system  or  SOS  must  be  made  after  the  design  is  complete  and  this  can  be  costly. 
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In  2003  the  Navy  began  implementing  eontinuous  proeess  improvement 
initiatives  using  Lean  Six  Sigma  (LSS)  to  eliminate  waste  and  identify  opportunities 
using  leading  indieators  to  determine  system  and  program  performanee.  About  2006  PEO 
C4I  initiated  a  number  of  proeess  improvement  events  to  reduee  the  total  ownership  costs 
of  acquiring  new  capability. 

In  2008  the  Deputy  Secretary  of  Defense  directed  all  services  and  activities  in 
DOD  to  begin  using  LSS.  SSC  LANT  production  and  the  CSRR  ISEA  began  using  LSS 
to  find  efficiencies  and  took  an  additional  step  to  obtain  capability  maturity  model 
integration  (CMMI)  level  three  certification  from  the  Software  Engineering  Institute  as 
well  as  meeting  International  Standards  Organization  (ISO)  9000/9001  standards. 

Since  2008  the  CSRR  program  completed  six  projects  and  SSC  LANT  another  15 
related  to  increasing  quality  and  reliability,  reducing  cycle  time,  improving  development 
first  pass  yields  and  achieve  costs  savings  and  avoidance.  The  CSRR  LSS  events 
performed  sponsored  by  the  program  office  are  listed  in  Table  22  with  the  objective  and 
the  outcome.  Several  of  these  were  directed  at  specific  systems  but  the  overall  results 
provided  benefits  to  CSRR  as  a  SOS.  Investigation  of  REDACS  and  BRR-6  identified 
gaps  affecting  capability,  logistics,  and  requirements  definition.  Another  confirmed 
planning  and  executing  installation  as  a  package  of  capabilities  could  deliver  cost  and 
schedule  benefits.  Implementing  continuing  process  improvement  demonstrates  it  is  a 
practice  which  both  a  systems  and  system  of  systems  should  attempt  to  achieve.  Effective 
process  improvement  also  identifies  leading  indicators  and  benchmarks  to  assess 
performance  of  the  SE  activities  and  the  overall  program  (Oppenheim,  Murman  and 
Secor  2009). 
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Table  22.  Lean  Six  Sigma  Projects  Completed  or  in  Progress 


LSS  Event 

Goal/Obiective 

Outcome/Comments 

CSRR  Value 
Stream 

Analysis 

Shorten  the  development 
cycle  and  identify  the  major 
activities 

Identified  all  of  the  activities  required  to 
create  a  CSRR  version.  Established  the 
development  timeline  at  24  months  and 
identified  $60M  in  cost  avoidance  for 
accelerating  the  EA  version  of  CSRR 

RFDACS 

Reliability 

Improvements 

Identify  the  root  cause  of 
RFDACS  failures  and 
develop  solutions 

Isolated  root  causes  of  several  problems. 
Developed  groom  procedures.  Increased 

Ao  to  .97  and  identified  $I0M  cost 
avoidance  for  the  fleet  in  reduced  repairs 

Testing  Cycle 
Reduction 

Eliminate  duplicative 
testing  performed  during 
SDVT  and  SAT 

Identified  $128K  in  savings  through 
eliminating  duplicative  testing 

BRR-6 

Reliability 

Improvements 

Identify  root  cause  of  poor 
buoy  performance  and 
develop  potential  solutions 

Identified  22  improvement 
recommendations  for  operating 
procedures,  technical  documentation,  and 
operator  and  maintenance  training. 
Developed  a  successful  case  to  the  CNO 
resource  sponsor  to  fund  improvements 
to  the  buoy 

Installation 

Cost  Sharing 

Productivity 

Improvement 

Capture  the  costs  of 
executing  consolidated  SOS 
installations 

Validated  consolidated  installations  are 
more  effective.  Identified  $2.5M  savings 
across  the  first  five  installations  on  LA 
platforms 
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4,  What  Lessons  Learned  Can  be  Applied  to  Future  Versions  of 
Common  Submarine  Radio  Room  and  Common  Radio  Room  for 
Surface  Combatants? 

Examination  of  the  case  studies,  systems  engineering,  system  of  systems 
engineering  and  integration  principles  and  acquisition  policies  identified  where  there 
have  been  successes  and  failures.  Further  understanding  the  history  of  how  submarine 
communications  has  increased  in  complexity  provided  context  in  looking  at  how 
individual  systems  evolved  into  a  SOS.  These  case  studies  were  able  to  capture  the 
lessons  learned  so  they  can  be  shared  with  others.  Examining  CSRR  using  the  Friedman 
and  Sage  framework  identified  many  of  the  learning  principles  noted  in  the  case  studies 
used  in  this  research.  Table  23  lists  the  learning  principles  identified  from  the  information 
available  regarding  CSRR.  The  SOS  principles  identified  would  be  applicable  to  other 
SOS  regardless  if  they  are  a  DOD  or  commercial  entity. 

One  important  observation  noted  is  the  fact  too  many  opportunities  are  allowed  to 
pass  where  a  case  study  would  be  of  value.  The  lessons  learned,  learning  principles,  or 
teachable  moments  could  be  captured  and  shared  with  others.  ESS  and  CMMI  events 
provide  an  opportunity  for  improvement  as  well  using  a  systematic  process  to  capture 
data  and  information  while  accomplishing  a  goal.  Maier  and  Rechtin  (2009)  highlights  a 
number  of  heuristics  which  are  applicable  to  systems  and  systems  of  systems. 

The  learning  principles  identified  in  Table  23  from  studying  CSRR  could  be 
applied  to  any  SOS.  Several  of  these  learning  principles  are  related  to  one  area  and 
discussed  in  more  detail  below.  These  lessons  can  be  shared  with  the  current  and  future 
development  of  CSRR  versions  and  are  applicable  to  any  system  of  systems.  These 
lessons  learned,  or  learning  principles  are  listed  below. 
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Table  23.  CSRR  Learning  Principles 


Integration  approach 

The  government  acts  as  the  integrator  and  program  manager 

A 

Requirements  definition  and 
management 

Clearly  define  the  requirements  and  write  the  requirements 
clearly  for  the  ENTIRE  life  cycle 

B 

Systems  architecture 
development 

Don't  begin  building  before  the  architecture  has  been  defined 
(or  don't  engi neer  just  for  the  sake  of  engi  neeri  ngl) 

C 

System/  subsystem  design 

Design  of  an  acknowledged  system  of  systems  must  be  shared 
to  the  maximum  extent  practicable. 

D 

Systems  integration  and 
interfaces 

Maintain  control  ofthesystem  ofsystems  at  the  interfaces 
from  a  physical,  functional,  and  logical  approach. 

E 

Verification/  validation 

Design  the  test  to  test  the  design  and  trust  but  verify 

F 

Deployment  and  post 
deployment 

Keep  your  customer(s)  in  mind 

G 

Life  cycle  support 

Account  for  all  of  the  "ilities"  when  developing  the  system  of 
systems  design 

H 

Risk  assessment  and 

management 

Expect  the  unexpected  and  embrace  change.  Ifs  inevitable 

1 

System  and  program 
management 

A.  Go  fastwhenever  possible,  otherwisego  slow 

B.  Perfect  is  the  enemy  of  good  enough 

C.  Be  a  trusted  partner  and  build  effective  relationships 

D.  Most  i mportantly  create  the  most  effective  team  possible  and 
keep  them  engaged,  motivated,  and  productive 

1,  Lesson  One:  Clearly  Define  the  Requirements  and  Write  the 
Requirements  Clearly  for  the  ENTIRE  Life  Cycle 

While  CSRR  is  a  system  of  systems  and  has  an  approved  set  of  requirements 
these  must  be  balanced  against  the  constituent  subsystems  to  ensure  the  requirements  of 
both  are  met.  Many  conflicts  arise  due  to  the  conflicting  requirements  between  program 
in  different  program  offices.  This  is  not  limited  to  just  within  the  individual  program 
offices  but  also  concerns  the  interactions  of  programs  managed  in  different  program 
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offices.  Over  70  pereent  of  a  program’s  eost  is  in  the  operations  and  sustainment  phase. 
This  ean  be  partially  mitigated  if  the  requirements  are  elear  and  do  not  eonfliet.  Clear 
requirements  with  some  elearly  established  flexibility  to  allow  for  evolving  systems  and 
eapabilities  should  be  considered.  System  of  systems  are  evolutionary  and  have  no 
defined  ending  date.  However  there  are  still  three  eriteria  that  should  be  met  when 
developing  or  evaluating  requirements:  (1)  They  must  be  preeise,  (2)  they  must  be 
verifiable,  and  (3)  they  must  be  traeeable  (Madni  and  Sievers,  2014,  43).  While  this 
applies  to  systems  level  it  is  even  more  important  for  the  SOS  engineer  sinee  many  times 
the  requirements  for  an  aeknowledged  system  ean  be  fuzzy  and  unelear.  Reframing  them 
in  eontext  of  the  eriteria  above  will  improve  the  probability  of  suoeessfully  building  the 
right  eapability. 

2,  Lesson  Two:  Do  Not  Begin  Building  before  the  Architecture  Has  Been 
Defined  (or  Do  Not  Engineer  Just  for  the  Sake  of  Engineering!) 

An  open  systems  arehiteeture  must  provide  as  mueh  flexibility  as  possible  to  the 
eonstituent  programs  while  maintaining  the  integrity  of  the  whole  system  of  systems. 
However  it  must  also  be  elearly  defined.  This  has  been  proven  painfully  true  on  more 
than  one  oeeasion.  The  F-111,  TBMCS  and  International  Spaee  Station  demonstrated 
attempting  to  build  something  without  fully  understanding  the  requirements  and 
speeifieations  invariably  lead  to  building  the  wrong  item  or  delivering  the  wrong 
capability.  The  lead  SOS  engineer  must  lead  the  team  to  define  the  SOS  arehiteeture  as 
eompletely  as  possible  in  order  to  ereate  and  deliver  the  eapabilities  needed.  This  period 
involves  a  lot  of  ereative  thinking  and  almost  no  bending  metal  or  turning  serews.  Clearly 
defining  the  arehiteeture  will  improve  the  odds  of  building  the  right  eapability.  The 
following  quote  from  Maier  and  Reehtin  (2009,  176)  “A  system  will  develop  and  evolve 
mueh  more  rapidly  if  there  are  stable  intermediate  forms  than  if  there  are  not”  often 
proves  true  when  working  with  what  is  known  to  be  factual  vice  what  is  undefined.  This 
might  sound  like  a  direet  SOS  is  the  eorreet  approaeh  but  this  applies  to  aeknowledged 
SOS  as  well. 
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3.  Lesson  Three:  Design  of  an  Acknowledged  System  of  Systems  must  be 
Shared  to  the  Maximum  Extent  Practicable 

Common  Submarine  Radio  Room  displays  many  of  the  eharaeteristics  of  an 
acknowledged  system  of  systems.  As  a  formal  program  CSRR  engages  with  other 
programs  to  develop  and  deliver  the  set  of  capabilities  needed  by  the  submarine  force.  In 
order  to  be  successful,  open  collaboration  of  information  is  paramount.  Sharing 
information  is  a  two-way  activity.  Constituent  systems  need  information  as  well  so  they 
can  attempt  to  meet  the  SOS  requirements.  Holding  onto  information  and  not  sharing  it 
undermines  the  integrity  of  the  SOS  and  the  ability  to  deliver  the  capability  to  the 
warfighter. 

4,  Lesson  Four:  Maintain  Control  of  the  System  of  Systems  at  the 
Interfaces  from  a  Physical,  Functional,  and  Logical  Approach 

One  characteristic  an  acknowledged  SOS  must  always  consider  is  how  much 
control  should  be  exerted  over  the  constituent  systems  and  their  interfaces.  However,  if 
there  is  no  manager  of  the  interfaces,  then  it  is  probably  not  an  acknowledged  SOS.  A 
Direct  SOS  would  have  full  and  complete  control  over  defining  and  directing  what 
interface  specifications  must  be  followed.  An  acknowledged  SOS  does  not  control  the 
other  programs  without  agreements  between  the  SOS  and  the  constituent  systems.  CSRR 
attempts  to  maintain  this  relationship  through  constant  engagement  with  the  constituent 
programs.  This  includes  working  with  new  programs  to  ensure  interoperability 
requirements  are  addressed,  and  determining  the  impact  of  changes  to  mature  programs. 
If  this  is  not  possible,  then  it  is  incumbent  for  the  SOS  to  assume  this  responsibility  and 
share  the  information  with  the  other  programs. 

One  specific  aspect  that  must  be  considered  is  when  the  interfaces  between 
systems  or  within  a  system  as  it  moves  from  one  configuration  to  the  next.  The  key  piece 
is  managing  the  interfaces  so  changes  within  the  individual  systems  will  not  perturb  the 
other  systems.  Maier  and  Rechtin  pointed  out  in  The  Art  of  Systems  Architecting  (2009), 
“The  greatest  leverage  in  system  architecting  is  at  the  interfaces.  The  greatest  dangers  are 
also  at  the  interfaces.” 
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5. 


Lesson  Five:  Go  Fast  Whenever  Possible,  Otherwise  Go  Slow 


Common  Submarine  Radio  Room  as  a  system  of  systems  requires  effeetive  and 
eollaborative  governanee  to  manage  delivery  and  sustainment  of  eapability.  This  is  not  a 
parable  related  to  the  tortoise  and  the  hare  but  more  aeeurately  the  heuristie  “haste  makes 
waste.”  This  normally  oecurs  when  event  based  and  ealendar  based  sehedules  eonfliet 
and  artifieial  timelines  are  ereated.  Few  people  like  to  admit  they  pad  a  sehedule  to  allow 
for  those  moments  when  they  eannot  work  on  their  project.  Parkinsons  law  states  “work 
expands  so  as  to  fill  the  time  available  for  its  completion”  and  the  student  syndrome  is 
“2/3  of  the  work  will  be  done  in  the  last  1/3  of  the  time”  (SPAWAR  2011;  Goldratt  1997, 
114-128).  These  relate  to  the  first  heuristic  in  there  is  always  a  normal  tendency  to 
address  the  crisis  of  the  day  and  put  off  a  task  until  it  is  close  to  or  at  a  crisis  stage.  Then 
we  make  haste  to  get  the  project  done  in  time  to  meet  the  deadline  which  is  typically 
missed  or  the  final  product  is  incomplete.  We  see  this  occur  almost  everywhere  we  look. 
The  problem  is  if  the  work  is  held  off  until  the  last  moment  the  next  heuristic  from 
Murphy  of  “if  anything  can  go  wrong  it  will”  kicks  in.  So  the  goal  is  to  achieve  as  much 
progress  as  possible  within  the  timeline  that  is  allotted  but  do  not  expend  energy  to 
complete  it  too  far  in  advance.  When  managing  a  SOS,  there  are  many  pieces  which  must 
be  tracked.  Maintaining  a  steady  drumbeat  with  reasonable  schedule  expectations  results 
in  fewer  crises  and  will  keep  the  program  development  on  track. 

6,  Lesson  Six:  Account  for  All  of  the  “ilities”  when  Developing  the 
System  of  Systems  Design 

Collaboration  is  key  to  ensuring  the  overall  system  of  systems  is  supported  for 
documentation,  training  and  parts.  When  looking  at  the  “ilities”  (reliability, 
interoperability,  maintainability,  availability,  usability)  from  an  acknowledged  SOS 
perspective  the  first  issue  that  will  be  apparent  is  the  different  approaches  each  program 
followed  to  develop  their  system.  The  higher  level  intent  was  met  but  the  result  is 
different  than  another  team.  The  primary  focus  for  an  SOS  is  interoperability.  If  the 
disparate  systems  cannot  communicate  with  each  other  it’s  just  a  collection  of 
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components.  The  other  ilities  must  be  balaneed  to  ensure  the  optimization  of  one  system 
does  not  oceur  at  the  sub-optimization  of  the  others. 

7.  Lesson  Seven:  Design  the  Test  to  Test  the  Design,  and  Trust  hut 
Verify 

Focus  the  testing  on  changes  made  to  the  system  of  systems.  When  building  a 
system  or  system  of  systems  a  means  to  perform  verifieation  and  validation  has  to  occur. 
One  of  the  lessons  from  the  TBMCS  is  the  requirements  were  unclear  which  in  turn 
prevented  effective  test  planning.  When  ereating  a  test  for  an  SOS  verify  the 
requirements  themselves  are  elearly  stated  and  testable  via  one  or  more  means.  When 
ereating  an  acknowledged  SOS,  additional  derived  requirements  will  frequently  be 
identified.  These  need  to  be  included  in  the  testing  proeess  and  verified.  In  many  cases 
the  ability  to  fully  test  the  SOS  would  require  too  much  time  and  resourees.  These 
requirements  need  to  be  articulated  as  possible  risks  and  a  plan  for  testing  them  following 
employment  of  the  SOS  eapabilities. 

Common  Submarine  Radio  Room  has  eneountered  this  on  several  occasions  when 
evaluating  a  system  or  component  for  integration  into  the  SOS.  The  testing  eriteria  were 
vaguely  stated  or  established  unrealistic  conditions.  One  sueh  example  involved  testing  a 
system  whieh  required  having  four  other  platforms  plus  a  shore  station  involved.  The 
only  problem  is  the  system  had  not  been  fielded  to  the  shore  stations  yet.  This  would  be  a 
valid  SOS  test  but  inappropriate  for  an  individual  system. 

8,  Lesson  Eight:  Expect  the  Unexpected  and  Embrace  Change,  It  is 
Inevitahle 

Risks  must  be  evaluated  to  determine  if  there  is  any  potential  for  emergence.  One 
of  the  SOS  characteristics  previously  discussed  was  emergence.  Emergenee  is  really  the 
unexpeeted  oecurring.  If  the  changes  to  the  SOS  are  well  defined  and  doeumented  the 
likelihood  of  emergence  occurring  should  be  low.  Antieipating  emergence  may  occur  can 
provide  the  opportunity  to  identify  alternate  paths.  CSRR  has  a  relationship  with  a 
number  of  programs  whieh  are  constantly  modernizing  their  systems.  Unless  a  elose 
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relationship  is  maintained  to  keep  abreast  of  their  activities,  the  likelihood  of  emergence 
happening  is  high.  Understanding  there  will  always  be  change  will  minimize  the  amount 
of  occasions  to  be  surprised. 

9,  Lesson  Nine:  Perfect  is  the  Enemy  of  Good  Enough 

This  is  one  topic  which  causes  much  angst  between  acquisition  and  engineering 
teams.  The  acquisition  team  objective  is  to  develop  and  procure  a  system  which  will  meet 
the  threshold  requirements  (a.k.a.  the  minimum  standards).  On  the  other  hand,  most 
engineering  teams  are  driven  to  achieve  the  objective  (a.k.a.  possibly  polishing  a 
cannonball).  Neither  goal  is  wrong  but  all  factors  must  be  considered  when  designing  and 
building  a  SOS.  Unless  it  is  a  directed  SOS,  the  lead  engineer  has  no  control  over  the 
individual  systems  requirements.  If  the  POR  is  in  development,  opportunities  exist  to 
ensure  the  threshold  and  objective  requirements  are  achievable. 

Programs  in  development  today  are  attempting  to  achieve  perfection  by 
establishing  the  threshold  and  objective  to  the  same  value.  From  a  SOS  perspective,  this 
becomes  unachievable  since  the  overall  performance  is  dependent  on  the  individual 
systems  performance.  Establishing  a  more  realistic  set  of  requirements  for  the  POR  and 
the  SOS  also  increase  the  probability  of  building  a  capability  that  is  good  enough. 
General  Patton  made  the  following  statement  which  sums  this  up  clearly  “A  good  plan 
violently  executed  now  is  better  than  a  perfect  plan  next  week.”  (NDP  6  1995,  24) 

10.  Lesson  Ten:  Be  a  Trusted  Partner  and  Build  Effective  Relationships 

One  of  the  most  important  characteristics  of  an  effective  lead  systems  or  system 
of  systems  engineer  is  not  the  fact  he  can  describe  the  technical  specifications  in  detail.  It 
is  the  ability  to  work  with  teams  from  the  individual  PORs  in  order  to  create  consensus 
on  a  common  goal.  This  is  not  a  skill  limited  to  systems  engineering.  The  effective  lead 
engineer  will  be  able  to  understand  the  overall  SOS  goals  and  objectives  and  frame  them 
from  a  system  of  systems  perspective  so  all  members  of  the  teams  can  understand.  Maier 
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and  Rechtin  (2009)  stated,  “If  a  system  requires  voluntary  eollaboration,  the  mechanism 
and  incentives  for  that  collaboration  must  be  designed  in.”  From  an  SOS  point  of  view 
this  becomes  critically  relevant. 

11.  Lesson  Eleven:  Keep  your  Customer(s)  in  Mind 

The  system  of  systems  must  be  able  to  collect  performance  data  to  determine 
what  changes  should  occur  to  support  the  mission  system  of  systems.  A  frequent  problem 
seen  in  many  SOS  is  the  great  deal  of  complexity  built  into  the  constituent  systems.  This 
may  be  okay  if  the  planned  operator  is  an  engineer  or  a  technician  but  presents  potential 
problems  when  the  operator  is  given  a  minimum  amount  of  training.  Whether  it  is  a 
system  or  a  SOS,  it  must  be  operable  by  an  operator  who  has  been  given  an  appropriate 
level  of  training.  For  example,  many  of  the  CSRR  technical  manuals  covered  all  of  the 
systems  within  CSRR  but  did  not  cover  the  interfaces  to  the  antennas.  Conversely,  the 
antenna  technical  manuals  covered  the  antennas  but  did  not  cover  the  interfaces  to  CSRR, 
creating  a  gap  where  no  documentation  existed  for  the  operator.  When  looking  at  the 
SOS  from  the  user’s  perspective,  it  is  advantageous  to  have  a  CONOPS  which  describes 
how  the  SOS  will  be  operated  and  maintained.  Collecting  SOS  performance  data  from  the 
users  can  be  used  to  develop  improvements  and  address  deficiencies. 

12,  Lesson  Twelve:  Most  Importantly  Create  the  Most  Effective  Team 
Possible  and  Keep  Them  Engaged,  Motivated  and  Productive 

Regardless,  if  this  is  an  engineering  or  management  or  academic  issue,  if  there  are 
no  people  on  the  team,  nothing  will  get  done.  CSRR  has  had  the  advantage  of  recruiting 
and  retaining  a  large  number  of  talented  engineers,  technicians,  testers,  logistics  and 
program  management  personnel.  Creating  synergy  within  the  teams  keeps  them  focused 
on  the  immediate  problems  while  not  ignoring  the  longer  term  issues  and  goals.  The 
International  Space  Station  case  study  pointed  out  maintaining  a  competent  and 
experienced  staff  for  over  20  years  is  a  challenge.  Getting  the  teams  to  get  engaged  and 
remain  engaged  can  be  a  challenge  especially  when  circumstances  prevent  scheduling 
face  to  face  events.  One  issue  noted  during  the  development  of  CSRR  V3  is  the  lack  of 
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many  of  face  to  face  meetings  due  to  the  federal  budget  ehallenges  prevented  a  lot  of 
interpersonal  engagement  whieh  occurs  when  teams  are  together.  When  working  with 
multiple  systems  the  ability  to  engage  and  remain  engaged  is  one  of  the  key  eomponents 
to  successfully  managing  a  SOS  program. 

There  is  one  last  pieee  of  this  question  which  still  needs  to  be  answered  in  terms 
of  the  scope  of  this  ease  study  looking  out  beyond  CSRR  V3.  The  most  reeent  aequisition 
program  baseline  (APB)  (PMW770  2011a)  extended  the  CSRR  program  out  to  FY  2030. 
Unlike  a  system  whieh  may  have  a  defined  modernization  plan  and  requirements  which 
are  fairly  statie,  a  SOS  is  tied  to  the  plans  of  the  individual  systems.  A  direeted  or 
eollaborative  SOS  may  require  less  negotiation  due  to  their  nature  but  aeknowledged 
SOS  will  always  be  in  a  series  of  negotiations  to  assure  ehanges  within  one  program  do 
not  break  a  capability  in  another.  Too  often  this  is  not  identified  until  it  is  delivered  to  the 
end  user.  Common  Submarine  Radio  Room  has  a  large  role  in  maintaining  effective  and 
open  communications  with  other  systems.  Over  time,  this  changes  as  people,  policies  and 
teehnology  come  and  go.  The  version  approaeh  is  planned  to  remain  in  place  but  will 
continue  to  evolve  as  well  to  meet  the  needs  of  the  stakeholders.  The  Ohio  SSBN 
replacement  will  deploy  some  version  of  CSRR,  whether  it  is  the  one  envisioned  today 
remains  to  be  seen.  For  the  other  domains  considering  using  the  CSRR  model  they  have 
mueh  to  eonsider  in  order  to  define  the  architecture  which  supports  their  platforms  and 
fits  within  the  larger  mission  SOS  role  defined  within  the  military  strategic  planning. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS  FOR  FURTHER 

INVESTIGATION 


A,  CONCLUSIONS 

The  C4I  capabilities  of  the  submarine  force  have  evolved  greatly  over  the  last 
century.  Expanding  use  of  the  RF  spectrum  and  introduction  of  new  technologies  led  to 
fielding  systems  of  increasing  complexity.  As  these  systems  were  integrated  into  larger 
systems  and  systems  of  systems  new  capabilities  emerged.  The  challenges  experienced 
by  DOD  with  the  integration  of  these  systems  led  to  questions  of  ownership 
responsibilities  of  these  new  capabilities  and  how  they  should  be  managed.  This  in  turn 
required  defining  a  SOS  and  their  characteristics.  Depending  on  the  type  of  SOS 
programmatic  and  systems  engineering  decisions  are  reached  which  may  not  be  in  the 
best  interests  of  the  SOS.  Even  today  this  is  a  major  issue  with  many  acknowledged  SOS 
created  within  DOD  having  minimal  oversight.  Only  recently  has  DOD  recognized  their 
acquisition  approach  must  shift  from  an  individual  systems  requirements  mentality  to  a 
mission  based  mentality  requiring  a  much  more  holistic  examination  of  what  is  needed  to 
achieve  a  given  capability. 

Systems  engineering  and  SOS  engineering  share  many  characteristics  but  the 
applications  differ  by  their  approach.  A  system  will  have  clearly  defined  requirements 
and  a  defined  life  cycle.  SOS  requirements  are  more  generalized  and  possess  an 
evolutionary  life  cycle  which  changes  but  does  not  end.  A  system  normally  has  a  single 
program  manager  whereas  depending  on  the  type  of  SOS  may  not  have  one  at  all.  Most 
SOS  within  DOD  are  considered  acknowledged  SOS.  Policy  guidance  from  OSD  is 
providing  the  framework  for  developing  and  managing  systems  of  systems.  Acquisition 
and  systems  commands  have  in  turn  recognized  many  of  their  products  can  be  classified 
as  a  SOS  or  are  a  constituent  component  of  a  system  of  systems.  This  can  be  seen  in 
Figure  33  by  looking  at  a  system  such  as  ADNS  supporting  a  C4I  system  of  systems  in 
CSRR  which  in  turn  supports  the  combat  systems  SOS  in  SWFTS  to  the  Virginia 
platform  system  of  systems  which  ultimately  supports  the  larger  mission  system  of 
systems. 
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Virginia  Submarine  Platform  POD  Mission  System  of  Systems 


Submarine  Warfare  Federated  Naval  Forces 

Tactical  System 

,  ,  Submarine  Force 

Common  Submarine  Radio  Room 

ADNS 

Figure  33.  Systems  to  System  of  Systems  Management  Perspectives  (after 
Director,  Systems  and  Software  Engineering  2008,  12) 

The  submarine  force  quickly  recognized  they  needed  to  leverage  the  capability  of 
these  systems  while  bounding  them  with  the  limitations  inherent  for  their  platforms, 
specifically  in  terms  of  space,  weight  and  power.  The  introduction  of  the  Trident 
integrated  radio  room  represented  the  first  step  toward  employing  a  contractor  furnished 
system  of  systems  capability.  The  submarine  communications  support  system  took  the 
next  step  by  introducing  automation  and  coordinated  installation  approaches.  Common 
Submarine  Radio  Room  is  the  culmination  of  these  efforts  while  introducing  open 
systems  architecture  designed  to  combine  and  leverage  its  constituent  systems  to  deliver 
capabilities  not  possible  in  an  individual  manner.  The  approach  for  developing  CSRR  has 
evolved  as  well,  moving  from  developing  a  specific  increment  version  for  each  class  to 
the  point  where  a  single  version  delivers  a  complete  core  capability  capable  of  accounting 
for  any  unique  platform  characteristics.  Clearly  defining  and  balancing  the  requirements 
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of  the  constituent  systems  composing  CSRR  within  the  system  of  systems  architecture 
means  attempting  to  optimize  one  system  over  the  others  can  be  detrimental  to  the  overall 
system  of  systems. 

The  development  of  case  studies  serves  several  purposes.  Case  studies  provide 
opportunities  to  capture  information  about  a  particular  event  or  system.  The  case  studies 
may  vary  in  their  approach  but  the  main  result  is  identifying  lessons  learned  or  learning 
principles.  NASA  and  the  Air  Force  consider  case  studies  to  be  a  valuable  means  for 
capturing  and  sharing  learning  principles  as  explicit  knowledge.  The  learning  principles 
identified  from  the  case  studies  confirmed  CSRR  would  make  a  viable  case  study.  The 
lack  of  available  C4I  case  studies  for  other  PEO  C4I  and  SPAWAR  systems  reinforced 
the  benefits  of  developing  a  case  study  involving  systems  managed  within  the  CSRR 
program. 

This  research  examined  the  question  if  CSRR  met  the  characteristics  to  be 
classified  as  a  system  of  systems.  The  SOS  characteristics  furthered  defined  CSRR  as  an 
acknowledged  SOS.  As  an  acknowledged  SOS  CSRR  have  requirements,  funding  and 
management.  These  must  be  balanced  with  the  other  systems  that  make  up  the  whole 
SOS.  System  changes  are  primarily  managed  by  the  parent  program  but  are  closely 
collaborated  with  the  CSRR  program  to  avoid  or  minimize  degradation  or  disruption  of 
capability.  As  a  system  of  systems,  CSRR  provides  redundancy  in  several  ways.  If  a 
communications  path  is  not  available  another  can  be  selected.  If  there  is  a  network 
failure,  alternate  means  to  reroute  or  restore  network  management  exist.  Additionally,  an 
examination  of  submarine  communications  demonstrated  the  evolutions  from  individual 
stove  pipe  systems  to  fully  integrated  and  interoperable  SOS  can  deliver  more  capability 
than  if  each  system  were  employed  separately.  The  research  identified  CSRR  was  not  a 
result  of  a  Manhattan  project  approach  but  rather  another  step  in  the  evolution  of 
submarine  communications. 

This  case  study  confirmed  systems  engineering  and  system  of  systems 
engineering  share  similar  qualities  but  are  applied  differently.  The  challenge  lies  in  the 
SOS  approach  that  is  implemented.  Most  DOD  SOS  are  considered  acknowledged  SOS 

due  to  each  individual  program  maintaining  its  own  program  and  funding  responsibilities. 
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The  net  result  is  these  systems  ereate  an  iterative  impact  on  other  systems  through 
introduction  of  new  capabilities,  phasing  out  old  ones,  changes  to  hardware  or  software, 
or  changing  operational  planning.  This  increased  emphasis  on  a  SOS  approach  means  a 
more  holistic  view  is  required  when  evaluating  a  new  SOS  or  one  that  is  already 
established.  The  guidance  promulgated  by  Director,  Systems  and  Software  Engineering 
(2008)  and  the  DASN  (RDTE)  2013  draft  provided  a  good  starting  point  to  begin 
implementing  SOSE  principles.  Specifically  the  seven  core  elements  a  SOS  engineer 
must  be  involved  in  encompass  translating  SOS  capability  objectives  into  SOS 
requirements  to  coordinating  and  monitoring  changes  to  improve  SOS  performance 
(Director,  Systems  and  Software  Engineering  2008,  92).  Another  thought  about  the 
difference  between  systems  engineering  and  SOS  engineering  is  the  level  of  complexity 
involved.  A  system  can  be  decomposed  into  its  discrete  components.  A  radio  can  be 
decomposed  to  a  power  supply,  amplifier,  modulator  and  demodulator.  A  SOS  considers 
the  systems  to  be  the  discrete  components.  This  changes  the  level  of  complexity  the  SOS 
engineer  must  consider  when  developing  or  changing  a  SOS. 

In  summary,  the  effective  application  of  SOS  engineering  principles  can  be 
applicable  to  a  variety  of  SOSs.  The  challenge  will  be  related  to  the  type  of  SOS  and  if 
there  is  a  clear  vision  of  what  the  SOS  must  be  able  to  do.  If  designing  a  hospital  the 
considerations  need  to  include  such  factors  as  the  location,  type  of  hospital,  services  to  be 
offered,  etc.  The  same  approach  can  be  taken  to  build  a  command  and  control  system.  Or 
they  can  be  applied  to  build  an  afloat  communications  architecture  similar  to  CSRR.  Case 
studies  provide  a  means  to  capture  these  lessons  learned  from  others  so  it  can  be  retained 
as  explicit  knowledge  and  shared  with  future  engineers,  technicians  and  managers.  In  the 
end  the  SOS  engineer  must  learn  from  the  experience  of  others  and  be  capable  of 
balancing  the  needs  of  the  systems  and  the  SOS. 

B.  RECOMMENDED  AREAS  OF  FURTHER  STUDY 

The  body  of  knowledge  regarding  systems  of  systems  is  beginning  to  expand  as 
recognition  just  how  closely  systems  are  tied  together  to  provide  new  capabilities.  This 
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case  study  examined  how  CSRR  met  the  eriteria  for  a  system  of  systems  and  the  lessons 
gained  from  the  program.  The  following  areas  could  possibly  merit  further  investigation. 

1 .  An  examination  to  develop  a  ease  study  of  PEO  SUBs  approaeh  to  the 
development  and  sustainment  of  SWFTS  would  add  to  the  knowledge 
base  for  the  SOS  community. 

2.  What  resources  exist  to  train  SOS  engineers?  A  study  was  performed  for 
systems  engineers  so  a  similar  study  ean  emphasize  the  differences  of  a 
system  and  a  SOS. 

3.  Investigate  what  leading  indieators  exist  for  measuring  the  performance  of 
a  SOS  and  how  can  they  be  utilized  to  provide  a  user  insight  to  potential 
degradations. 

4.  An  examination  of  the  SOS  approaeh  to  the  DOD  cloud  development 
projects 

5.  An  assessment  of  implementing  a  quality  function  deployment  approach 
to  evaluate  the  benefits  or  risks  of  making  changes  to  a  SOS. 
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